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\  c 

This  report  and  its  executive  summary^ explain  why  and  how  the 
Air  Force  would  benefit  from  major  changes  in  how  it  acquires  and 
supports  aviation  electronics  (avionics)  equipment.  This  report 
describes  an  integrated  strategy  for  implementing  such  reform,  and  it 
examines  the  rationale  upon  which  the  strategy  is  founded.  The  rea¬ 
sons  for  reform  have  been  building  fortwenty  years,  as  witnessed  by  a 
continuing  stream  of  RAND  researclrall  sponsored  by  the  Air  Force 
and  often  with  direct  special  assistance  from  operational  units.3" 

A  strategy  for  reforming  the  avionics  acquisition  process  by 
rearranging  avionics  development  responsibilities  was  proposed^  in  -  c' 

D.  W.  Mclver,  A.  I.  Robinson,  and  H.  L.  Shulman,  with  W.  H. 

Ware,  Proposed  Strategy  for  the  Acquisition  of  Avionics  Equipment, 
\  The  RAND  Corporation,  R-1499-PR,  December  1974. 

Although  this  strategy  was  partly  carried  out,  the  controversy  over  its 
main  elements  led  the  Air  Force  to  adopt  other  measures,  such  as  the 
1978  creation  of  a  Deputy  for  Avionics  Control  (DAC),  with  responsi¬ 
bility  for  controlling  avionics  acquisition  but  lacking  direct  authority 
over  both  budgeting  and  program  management.*^' 

As  the  DAC  was  being  established,  RAND  was  researching  alterna¬ 
tives  for  addressing  the  deficiencies  in  the  support  process  for  the 
F-15’s  avionics  equipment.  Air  Force  actions  resulting  from  this  work 
included  the  procurement  and  deployment  of  $150  million  of  additional 
test  equipment  for  the  avionics  intermediate  shops,  and  the  procure¬ 
ment  of  additional  spares  for  the  avionics. 

The  need  for  these  procurements  was  briefed  to  General  Alton  Slay, 
who  was  then  Commander  of  the  Air  Force  Systems  Command,  during 
September  1979.  He  decided  to  sponsor  a  new  RAND  project  on 
Avionics  Acquisition  and  Support.  This  report  and  its  companion 

'Gebman  and  Shulman,  1988. 

2Selected  reports  on  the  early  research  include:  Robinson  and  Shulman,  1967,  1972. 
Nelson  et  al.,  1974. 

3  Assisting  Air  Force  units  have  been  stationed  at  such  air  bases  as  Bitburg,  Cannon, 
Camp  New  Amsterdam,  Hahn,  Hill,  Holloman,  Langley,  Myrtle  Beach,  Pease,  and 
Plattsburg.  They  have  operated/supported  such  aircraft  as  the  F-4C,  A-7D,  F-211A, 
F-111D,  FB-111,  F-15A,  F-15C,  and  F-16A. 

4The  DAC  does,  however,  report  both  to  the  Air  Force  Systems  Command  through 
the  Air  Force  Aeronautical  Systems  Division  and  to  the  Air  Force  Logistics  Command; 
such  reporting  appears  to  be  a  main  method  for  influencing  avionics  acquisition  deci¬ 
sions. 
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executive  summary  constitute  the  final  report  for  that  project. 
RAND’s  initial  charter  was  to  assess  the  Air  Force’s  progress  with  such 
measures  as  the  DAC  and  to  suggest  further  steps  the  Air  Force  might 
take  to  improve  the  avionics  acquisition  and  support  processes.  To  do 
so,  RAND’s  effort  initially  consisted  of  two  phases: 

Phase  I.  Assess  progress  to  date  as  manifested  by  the  performance 
of  fielded  equipment  and  identify  areas  of  needed 
improvement. 

Phase  II.  Define  and  examine  alternative  ways  of  achieving  needed 
improvements. 


Phase  I  work  raised  serious  concerns  about  the  ability  of  maintenance 
personnel  to  identify  and  fix  faults  that  maintenance  records  suggested 
may  be  persisting  for  we-?ks  and  even  months.  Phase  II  work  led  to 
the  recommendation  that  the  Air  Force  institute  a  special  and  separate 
phase  of  development  (termed  maturational  development).  During 
Stage  1  (assessment),  the  government  would  contract  directly  with  the 
weapon  system  prime  contractor,  as  well  as  selected  subsystem  contrac¬ 
tors,5  for  four  engineering  services: 

1.  Fielding  a  joint  team  to  collect  detailed  R&M  information 
from  one  or  more  operational  units. 

2.  Performing  engineering  analyses  to  define  the  most  serious 
deficiencies  in  R&M. 

3.  Defining  and  analyzing  alternatives  for  dealing  with  the  most 
serious  deficiencies. 

4.  Working  with  the  government  to  define  an  appropriate  and 
comprehensive  package  of  improvements. 

During  Stage  2  (implementation)  the  Air  Force  would  use  the  results  of 
such  services  to  initiate  integrated  improvement  programs  aimed  at  the 
most  cost  beneficial  improvements. 

RAND’s  1980  briefing  of  Phase  I  and  Phase  II  final  results  raised 
two  controversial  issues: 

•  What  was  the  real  condition  of  the  sophisticated  avionics  equip¬ 
ment  being  used  in  the  field?6 

5Selected  subsystems  would  be  the  complex  types  that  are  known  to  be  difficult  to 
mature  (radar,  weapon  delivery,  electronic  warfare,  etc.). 

6Fire  control  radars  on  the  F-15A  and  F-16A  appeared  to  be  experiencing  serious  dif¬ 
ficulties  with  reliability  or  maintainability.  All  the  squadrons  studied  had  many  aircraft 


•  Would  ongoing  Air  Force  activities  make  it  unnecessary  to 
invest  scarce  time  and  funds  in  a  maturational  development 
phase  for  such  avionics? 

The  first  issue  was  fueled  by  well-known  problems  with  the  accuracy  of 
the  Air  Force’s  standard  data  collection  systems.  The  second  was 
fueled  by  the  concerns  of  some  that  instituting  a  maturational  develop¬ 
ment  phase  would  be  costly,  could  lengthen  the  development  process, 
might  retard  the  incorporation  of  new  technologies,  and  could  resurrect 
old  arguments  about  an  even  larger  rearrangement  of  responsibilities 
for  avionics  development. 

During  the  fall  of  1980,  the  Air  Force  Systems  Command  Director¬ 
ate  for  Plans  decided  that  RAND  and  the  Air  Force  would  undertake 
two  measures  to  help  resolve  these  issues: 

•  RAND  would  extend  its  research  to  include  a  Phase  III  that 
would  more  thoroughly  research  the  real  condition  of  the 
fielded  radars  with  the  help  of  information  to  be  collected  by 
the  radar  contractors. 

•  The  F-15  and  F-16  radar  contractors,  along  with  the 
corresponding  weapon  system  prime  contractors,  would  be  given 
government  funded  opportunities  to  apply  data  collection  and 
engineering  analysis  methods  like  those  needed  for  matura¬ 
tional  development.  Within  the  Air  Force  this  effort  became 
known  as  the  F-15/F-16  Radar  R&M  Improvement  Program. 

These  two  measures  would  provide  RAND  an  opportunity  to  further 
assess  the  need  for  a  formalized  maturational  development  phase 
before  publishing  the  project’s  final  report. 

During  June  of  1981,  the  Deputy  Chief  of  Staff  for  Research, 
Development,  and  Acquisition  approved  the  concept  for  the  F-15/F-16 
Radar  R&M  Improvement  Program.  However,  the  contractor  teams 
did  not  start  collecting  data  until  June  1984.7 

with  radars  that  required  much  more  maintenance  than  other  aircraft.  These  data  (sum¬ 
marized  in  Appendix  A)  raised  the  disturbing  and  controversial  implication  that  mainte¬ 
nance  personnel  could  not  promptly  fix  certain  radar  problems.  Unfortunately,  subse¬ 
quent  in-depth  data  collection  and  analysis  by  the  radar  contractors  confirmed  this 
implication.  Section  IV  summarizes  the  contractors’  findings. 

7The  chief  activities  that  consumed  the  three  years  included:  (1)  coordination  among 
Air  Force  organizations  and  review  of  the  need  for  a  special  data  collection  effort  (12 
months),  (2)  preparation  of  a  Program  Management  Directive  (six  months),  and  (3)  the 
contracting  process,  including  preparation  of  Requests  for  Proposals  and  negotiation  of 
Memoranda  of  Understanding  among  the  participating  Air  Force  organizations:  three 
System  Program  Offices,  two  Air  Logistics  Centers,  and  four  air  bases  (18  months). 
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One  year  later  the  contractors  presented  findings  and  recommenda¬ 
tions  to  the  Aeronautical  Systems  Division  (ASD)  Strike  Systems  Pro¬ 
gram  Office  (SPO),  which  had  been  designated  as  the  program  manager 
for  the  contractor  efforts.8  The  Strike  SPO  and  the  cognizant  Air 
Logistics  Centers  (Ogden  for  the  F-16  radar  and  Warner  Robins  for  the 
F-15  radar)  then  briefed  results  and  recommendations  to  appropriate 
Air  Force  organizations  during  late  1985  and  early  1986. 

Drawing  from  contractor  results  and  the  Air  Force  briefings,  RAND 
completed  its  Phase  III  research  during  which  it: 

•  Further  assessed  the  condition  of  the  subject  radars  as  they  are 
used  in  the  field. 

•  Further  assessed  the  need  for  a  formalized  maturational 
development  phase  during  avionics  acquisition. 

•  Considerably  revised  its  proposed  strategy  for  improving  avion¬ 
ics  acquisition  and  support. 

The  results  of  this  Phase  III  research  have  been  combined  with  per¬ 
tinent  results  from  the  first  two  phases  to  produce  this  final  report  for 
the  Avionics  Acquisition  and  Support  project.  This  report  therefore 
summarizes  results  of  research  spanning  a  seven  year  period. 

Although  these  results  stem  from  research  directed  toward  the  more 
complex  avionics  subsystems  for  fighter  airplanes,  the  armed  services 
may  also  consider  applying  the  resulting  strategy  to  the  acquisition  and 
support  of  complex  electronics  in  other  mobile  military  systems  such  as 
bombers,  helicopters,  and  tanks. 

All  three  phases  of  RAND’s  work  were  accomplished  within  the 
Project  AIR  FORCE  Resource  Management  Program,  first  under  the 
Avionics  Acquisition  and  Support  project,  then  as  a  special  assistance 
effort  under  the  Resource  Management  Program's  Concept  Develop¬ 
ment  and  Project  Formulation  project,  and  finally  under  a  project  on 
Meth  ids  and  Strategies  for  Improving  Weapon  System  Reliability  and 
Maintainability,  which  was  sponsored  by  the  Air  Force  Special  Assis¬ 
tant  for  Reliability  and  Maintainability. 

At  the  time  of  publication,  several  of  the  radar  R&M  improvements 
identified  in  this  report  have  been  implemented,  others  are  under 
development  and  test,  and  some  are  planned  to  enter  development 
later.  Moreover,  several  ASD  SPOs  now  have  plans  to  use  various 
adaptations  of  the  maturational  development  concept  in  their  develop¬ 
ment  programs.  Also,  the  Air  Force  Deputy  Chief  of  Staff  for  Logistics 

8RAND  assisted  the  Strike  SPO  in  defining  the  contractor  efforts,  monitoring  prog¬ 
ress,  and  reviewing  results.  The  Strike  SPO  also  hired  a  contractor,  Support  Systems 
Associates,  Inc.,  to  further  help  oversee  the  equipment  contractor  efforts. 
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and  Engineering  is  reviewing  a  potential  use  of  maturational  develop¬ 
ment  to  form  the  backbone  of  component  improvement  programs  for 
avionics  subsystems. 


SUMMARY 


For  combat  critical  avionics,1  the  Air  Force  continues  to  have  prob¬ 
lems  maintaining  the  full  measure  of  designed  capabilities  essential  to 
sustaining  combat  superiority  into  the  next  century.  The  problems 
result  from  weaknesses  in  the  processes  that  the  Air  Force  uses  to 
acquire  and  support  avionics.  These  processes  have  been  weakened  by 
a  rapid  growth  in  the  complexity  of  avionics  that  has  been  accom¬ 
panied  by  a  failure  to  adapt  them  to  the  growing  challenges  of  acquir¬ 
ing  and  supporting  such  equipment.  This  report  aims  to  stimulate  con¬ 
sideration  of  a  major  reforming  of  these  processes. 

Section  I  reviews  current  problems  of  supporting  modern  avionics. 
Section  II  identifies  the  underlying  weaknesses  responsible  for  them 
and  then  describes  a  coherent  strategy  for  reforming  these  processes. 
The  major  proposals  constituting  the  strategy  are  developed  in  Secs.  Ill 
through  VI.  Although  these  proposals  are  derived  almost  exclusively 
from  RAND  research  on  avionics  equipment  for  fighter  airplanes  such 
as  the  F-15  and  F-16,  they  should  prove  beneficial  to  any  aircraft  that 
rely  heavily  on  sophisticated  avionics  to  perform  their  wartime  mis¬ 
sions,  and  to  such  other  mobile  military  systems  as  helicopters  and 
tanks  with  complex  electronics. 


PROBLEMS  OF  SUPPORTING  MODERN  AVIONICS 

Avionics  equipment  rarely  experiences  total  failure.  Rather,  it  typi¬ 
cally  falls  victim  to  faults  that  erode  its  performance  superiority  over 
potential  enemy  weapons.  When  a  piece  of  equipment  fails  to  deliver 
its  full  measure  of  designed  performance,  the  performance  degradation 
is  often  subtle  and  difficult  to  observe. 

Faults  manifest  symptoms  only  in  specific  operational  modes. 
A  subsystem  with  multiple  modes,  such  as  fire  control  radar,  can  have 
a  fault  that  affects  performance  in  only  certain  applications  of  particu¬ 
lar  modes.  Other  faults  show  symptoms  only  in  specific  environ¬ 
ments,  such  as  in  an  aircraft  that  is  executing  a  violent  maneuver. 
Thus,  it  is  important  to  distinguish  between  faults  we  term  Type  A  and 
Type  B: 


’The  term  avionics  is  used  here  to  refer  to  all  aviation  electronics,  including  airborne 
electronic  warfare  equipment. 
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•  Type  A  faults  have  stationary  observability.  They  mani¬ 
fest  symptoms  that  are  observable  no  matter  when  or  where  the 
equipment  is  operated  or  tested.  A  broken  picture  tube  in  a 
video  display  is  a  Type  A  fault. 

•  Type  B  faults  have  nonstationary  observability.  They 
manifest  symptoms  some  of  the  time.  A  faulty  connection  can 
be  a  Type  B  fault. 

The  current  avionics  acquisition  and  support  processes  concentrate 
mainly  on  Type  A  faults,  which  dominated  during  the  early  days  of  avi¬ 
ation  electronics.  Now,  however,  Type  B  faults  dominate  because 
greatly  improved  reliability  has  combined  with  larger  and  more  highly 
integrated  avionics  systems.  Type  B  faults  seriously  frustrate  the  iden¬ 
tification  and  correction  of  faulty  avionics  equipment.  Equipment  with 
such  faults  all  too  often  circulates  between  shop  and  airplane,  degrad¬ 
ing  the  plane’s  capabilities,  until  it  either  acquires  a  more  easily 
observable  fault  or  maintenance  technicians  set  it  aside  for  special 
attention. 


WEAKNESSES  IN  THE  ACQUISITION  AND 
SUPPORT  PROCESSES 

The  current  acquisition  process  actually  hinders  the  avionics 
developer  when  he  attempts  to  solve  the  extraordinarily  difficult 
engineering  challenges  associated  with  Type  B  faults.  The  main 
weaknesses  in  the  acquisition  process  are  the  need  for: 

•  Engineering  data  on  developing  technologies  and  equipment. 

•  A  corporate  memory  for  tracking  conditions  under  which  and 
reasons  why  technologies  fail. 

•  Adequate  time  to  develop  and  mature  equipment. 

•  Information  on  support  problems  with  new  equipment. 

•  Better  timing  of  transfer  of  management  responsibility  for 
equipment. 

The  current  support  process  similarly  hinders  avionics  technicians 
when  they  attempt  to  solve  the  difficult  maintenance  challenges  associ¬ 
ated  with  Type  B  faults.  The  main  weaknesses  in  the  current  support 
process  identified  in  Sec.  II  are  the  need  for: 

•  Adequate  information  about  performance  degradations  during 
routine  training  flights. 

•  A  system  for  tracking  avionics  equipment  performance  by  serial 
number  to  identify  equipment  with  problems  eluding  repair 
efforts. 
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•  Feedback  to  compensate  for  the  use  of  different  tests  and 
pass/fail  criteria  at  each  maintenance  level  (flightline,  shop, 
and  depot). 

•  Capabilities  for  avionics  technicians  to  repeat  certain  tests  and 
initiate  other  tests  before  thermal  equilibrium  is  achieved. 

•  An  adequate  procedure  for  techniciai  i  to  report  avionics  sup- 
portability  problems  and  recommend  possible  solutions. 


A  STRATEGY  FOR  REFORM 

The  strategy  proposed  for  reforming  avionics  acquisition  and  sup¬ 
port  examines  the  major  fundamental  weaknesses  and  the  most 
promising  solutions  that  have  come  to  the  attention  of  the  authors  dur¬ 
ing  20  years  of  research  in  this  field.  Recent  research  has  focused  on 
avionics  for  such  fighter  airplanes  as  the  F-15  and  the  F-16.  The  main 
source  of  data  was  the  special  data  collection  effort  conducted  during 
the  assessment  stage  of  the  F-15/F-16  Radar  R&M  Improvement  Pro¬ 
gram. 

The  two  goals  that  guided  formulation  of  the  proposed  strategy  are 
to  strengthen: 

•  The  reliability  and  maintainability  (R&M)  of  mission-essential 
avionics  equipment  already  in  the  field. 

•  The  development  process  so  that  future  generations  of 
mission-essential  avionics  equipment  can  more  regularly  deliver 
their  full  designed  capabilities. 

To  help  the  Air  Force  achieve  these  goals,  the  recommended  strategy 
consists  of  six  major  proposals  (Fig.  S.l). 

Proposal  1:  Accelerate  R&M-Related  Avionics  Technologies 

The  first  line  of  defense  for  R&M  is  for  engineers  to  design  it  right 
from  the  outset.  However,  engineers’  ability  to  do  this  is  influenced 
greatly  by  their  accumulated  knowledge  regarding  the  capabilities  and 
limitations  of  the  technologies  used  in  a  design.  Unfortunately,  to 
deliver  a  subsystem  design  with  the  specified  levels  of  functional  per¬ 
formance,  engineers  often  must  apply  the  latest  advances  in  technol¬ 
ogy.  For  emerging  technologies  that  are  critical  to  achieving  mission 
essential  performance,  it  is  therefore  vital  to  accumulate  a  body  of 
engineering  knowledge  as  rapidly  and  as  efficiently  as  possible  to  form 
a  solid  design  basis.  The  Air  Force  can  help  strengthen  the  engineer¬ 
ing  knowledge  available  to  designers  by  accelerating  the  development  of 
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key  avionics  technologies.  Certain  technologies,  moreover,  promise 
especially  important  benefits  to  R&M.  These  need  to  be  searched  out 
and  given  appropriate  emphasis.  To  help  do  this,  we  propose  accelera¬ 
tion  of: 


•  Development  of  selected  functional  performance  technologies. 

•  Research  on  failure  pathologies. 

•  Research  and  development  of  built-in  test  technologies. 

Although  the  Air  Force  already  has  efforts  in  each  of  these  areas,  there 
are  considerable  opportunities  to  increase  their  level. 

Proposal  2:  Improve  the  Ability  to  Test  Avionics  Equipment 

In  the  airplane,  on  the  flightline,  in  the  shop,  and  at  the  depot,  the 
Air  Force  needs  improved  abilities  to  test  avionics.  Even  with  the  best 
of  efforts  to  accelerate  the  advancement  and  application  of  R&M  tech¬ 
nologies,  faults  will  inevitably  develop  in  avionics  equipment.  The 
ability  to  test  and  find  faults  is  key  to  maintaining  designed  capabili¬ 
ties. 

To  improve  the  testability  of  avionics  equipment,  the  Air  Force  can 
place  greater  emphasis  on  improving  fault-isolation  capabilities  in  its 
research  and  development  programs  and  it  can  require  specific  types  of 
improvements  in  test  capabilities. 

Proposal  3:  Provide  More  Complete  Feedback 
on  Equipment 

Even  with  the  best  of  test  capabilities,  some  faults  will  inevitably 
escape  detection  by  ground  support  equipment.  Thus,  R&M  requires 
that  maintenance  personnel  have  timely  and  reasonably  complete  feed¬ 
back  to  deal  quickly  and  effectively  with  faulty  assets  that  escape 
repair. 

To  better  provide  such  information,  the  Air  Force  can  improve  the 
quality  of  information  received  from  the  pilot’s  postflight  debriefing  to 
maintenance  technicians  and  it  can  improve  the  technician’s  capability 
to  track  and  identify  hard-to-fix  faults. 

Proposal  4:  Adopt  a  Maintainability  Indicator 

Even  with  the  best  of  feedback  to  maintenance  technicians  about 
problems  with  equipment  performance,  R&M  needs  to  attract  manage¬ 
ment  attention  to  resolve  the  more  serious  maintainability  problems. 
Such  attention  is  essential  for  identifying  and  fixing  the  underlying 
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root  causes  of  problems.  It  is  also  essential  for  effectively  communicat¬ 
ing  maintainability  problems  to  the  research  and  development  com¬ 
munity.  It  is  important  to  avoid  repetition  of  maintainability  problems 
in  the  development  of  new  equipment. 

A  single  measure  to  indicate  the  overall  maintainability  of  a  subsys¬ 
tem  and  its  associated  ground  support  system  would  be  desirable.  Such 
an  indicator  could: 

•  Complement  the  existing  reliability  indicator  (MTBF)2  and 
with  it  provide  a  meaningful  composite  picture  of  equipment 
R&M. 

•  Be  sensitive  to  the  full  range  of  problems  that  arise  in  identify¬ 
ing  faults  and  isolating  their  causes. 

•  Account  for  all  flights  with  indications  of  faulty  subsystem 
operation. 

Proposal  5:  Institute  Maturational  Development 

Even  with  the  best  of  implementations  for  the  preceding  proposals, 
some  significant  R&M  problems  will  inevitably  evade  early,  satisfactory 
resolution,  especially  Type  B  faults  in  large,  complex,  and  tightly 
integrated  subsystems  incorporating  many  new  technologies.  Fire  con¬ 
trol  radars  for  fighter  airplanes  fall  in  this  category.  Such  subsystems 
and  their  associated  ground  support  systems  need  a  development  phase 
to  mature  their  R&M  to  the  levels  that  will  allow  the  subsystems  to 
regularly  deliver  the  full  measure  of  performance  for  which  they  were 
designed.  Such  a  maturational  development  phase  needs  two  stages: 

•  Stage  1,  Assessment:  Collection  and  analysis  of  engineering 
data  while  the  subsystem  is  in  normal  use  by  the  operator,  fol¬ 
lowed  by  analysis  of  candidate  improvements  and  formulation 
of  a  comprehensive  package  of  improvements3 

•  Stage  2,  Implementation:  Putting  into  operation  the  most 
cost-effective  improvements  that  aim  at  regular  delivery  of  full 
design  performance. 

Such  a  process  offers  a  further  line  of  defense  that  is  needed  to  assure 
the  delivery  of  necessary  R&M  characteristics  for  the  most  complex 
avionics  subsystems.  It  would  be  an  important  supplement  to  the 
measures  suggested  by  the  foregoing  proposals. 

2Mean  Time  Between  Failure. 

3Areas  of  needed  improvement  may  include  airborne  equipment,  ground  support 
equipment,  hardware,  software,  and  maintenance  procedures. 
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Proposal  6:  Reorganize  Avionics  Engineering  Resources 

By  reorganizing  its  avionics  engineering  resources,  the  Air  Force  can 
coordinate  implementation  of  activities  to  better: 

•  Accelerate  R&M  related  technologies  (especially  built-in  test 
technology)  to  provide  a  better  technical  base  for  new  develop¬ 
ment  programs. 

•  Incorporate  improved  test  capabilities  in  new  development  pro¬ 
grams. 

•  Track  equipment  performance  following  repair. 

•  Assess  maintainability  needs. 

•  Mature  equipment  R&M. 

In  contrast  to  such  finely  tuned  coordination,  past  avionics  research 
and  development  efforts  have  suffered  from  lack  of  R&M  guidance  to 
laboratory  projects,  a  potent  sponsor  for  the  advanced  development  of 
critical  elements,  an  agency  dedicated  to  supervising  maturational 
development,  and  an  agency  with  a  robust  engineering  organization 
overseeing  post-PMRT4  maturation  of  both  airborne  and  ground  sup¬ 
port  equipment. 

One  way  that  the  Air  Force  could  address  such  deficiencies  is  to 
reorganize  its  avionics  engineering  resources  in  the  form  of  an  Avionics 
Engineering  Center.  Such  a  center  would  help5  oversee  research, 
development,  and  maturation  of  sophisticated  avionics  subsystems  (fire 
control  radars,  electronic  warfare  systems,  and  the  like)  by 

1.  Sponsoring  advanced  development  of  critical  elements. 

2.  Starting  FSED  early  for  critical  subsystems. 

3.  Supervising  maturational  development  for  critical  subsystems. 

4.  Overseeing  post-PMRT  maturation  and  engineering  support. 

In  the  past,  there  has  been  resistance  to  such  concepts  as  an  Avionics 
Engineering  Center  and  subsystem -focused  maturational  development. 
Commonly  cited  reasons  include  concerns  about  the  adequacy  of  the 
Air  Force’s  resources  that  could  be  allocated  to  managing  subsystem 
development  and  maturation,  especially  in  areas  with  complex  inter¬ 
faces.  Other  concerns  are  based  upon  fears  that  the  application  of 

4Program  Management  Responsibility  Transfers  from  the  AF  Systems  Command  to 
the  AF  Logistics  Command  when  AFSC  completes  its  management  of  the  Full  Scale 
Engineering  Development  (FSED)  of  the  equipment. 

5Primary  responsibility  for  managing  subsystem  development  may  reside  with  a 
weapon  system’s  prime  contractor,  or  its  System  Program  Office  (SPO),  or  an  Avionics 
Engineering  Center.  In  the  first  two  situations,  the  Center  would  assist  the  cognizant 
SPO.  For  further  details  see  Sec.  VI. 
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technological  advances  may  be  retarded,  and  the  prime  contractors’ 
ability  to  optimize  an  overall  weapon  system  may  be  seriously  con¬ 
strained.  Against  such  concerns  the  Air  Force  needs  to  weigh  the  rela¬ 
tionship  between  past  practices  and  current  difficulties  with  avionics 
R&M  and  the  new  challenges  that  will  flow  from  the  rapidly  changing 
character  of  avionics  technology. 


CONCLUSIONS 

For  its  combat  airplanes  to  meet  and  defeat  future  threats,  the  Air 
Force’s  avionics  subsystems  will  continue  to  grow  in  terms  of  functions, 
sophistication,  and  complexity,  while  relying  heavily  on  the  latest 
advances  in  technology.  To  meet  the  challenge  of  acquiring  and  sup¬ 
porting  the  designed  capabilities  of  such  equipment,  the  Air  Force 
needs  to  aspire  to  even  higher  levels  of  excellence  in  avionics  R&M. 
The  Air  Force  would  benefit  from  the  kind  of  improved  coordination 
that  an  Avionics  Engineering  Center  could  provide  for  managing  the 
multiple  lines  of  defense  necessary  for  excellence  in  R&M.  Such  a 
center  could  be  the  capstone  to  a  major  reform  built  upon  four  corner¬ 
stone  concepts: 

•  Explicit  recognition  of  Type  B  faults 

•  Implementation  of  improved  debriefing  and  improved  tracking 
of  equipment  performance  after  maintenance  to  more  rapidly 
identify  Type  B  problems 

•  Use  of  fault  removal  efficiency  as  a  management  indicator  for 
maintainability 

•  Institutionalization  of  maturational  development  as  a  last  line 
of  defense  for  R&M. 

Although  any  of  these  concepts  would  prove  beneficial,  implementation 
of  the  complete  strategy  would  constitute  a  major  reform  to  how  the 
Air  Force  acquires  and  supports  avionics.  With  such  reform,  the  Air 
Force  could  exploit  rapidly  advancing  electronics  technologies  to  help 
ensure  the  continued  superiority  of  its  combat  airplanes. 
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I.  INTRODUCTION 


The  next  decades  will  see  growing  demands  placed  on  pilots  of  com¬ 
bat  airplanes.  To  acquire,  track,  and  destroy  the  highest  priority  tar¬ 
gets,  they  will  need  weapon  systems  that  can  sustain  a  margin  of 
superiority  over  the  equipment  of  prospective  enemies.  The  Air  Force 
long  ago  set  a  trend  of  relying  on 

•  More  avionics  equipment, 

•  Greater  sophistication  of  individual  avionics  subsystems,  and 

•  Increased  interdependence  among  avionics  subsystems. 

This  trend  unfortunately  also  increases  the  challenge  of  providing  high 
reliability  and  easy  maintenance  for  these  avionics  subsystems.  The 
combat  effectiveness  of  such  high  performance  equipment  is  sustained 
only  by  coupling  high  functional  performance  with  high  reliability  and 
ease  of  maintenance. 

This  report  analyzes  the  reasons  why  the  Air  Force  needs  to  make 
major  reforms  in  how  it  acquires  and  supports  avionics  equipment.  It 
then  formulates  a  cohesive  strategy  for  accomplishing  such  changes. 
Both  the  analysis  of  the  current  situation  and  the  formulation  of  a 
strategy  for  improvement  were  guided  by  two  main  goals: 

•  To  improve  the  reliability  and  maintainability  of  mission- 
essential  avionics  equipment  already  in  the  field  so  that  it  may 
more  regularly  deliver  its  designed  capabilities. 

•  To  strengthen  the  development  process  so  that  future  generations 
of  mission-essential  avionics  equipment  can  more  regularly 
deliver  their  full  designed  capabilities. 


CHALLENGES 

The  Air  Force’s  challenge  to  provide  high  performance  avionics 
equipment  that  dependably  delivers  the  full  measure  of  designed  per¬ 
formance  will  increase  simply  because  potential  enemies,  especially  the 
Soviet  Union  and  its  allies,  pose  greater  and  greater  threats.  They  out¬ 
number  us  in  military  personnel  and  weapons.  In  addition,  the  techno¬ 
logical  sophistication  and  functional  performance  of  their  weapons  are 
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constantly  improving.1  Because  the  future  will  see  potential  enemies 
exploiting  technological  advances  to  enhance  their  war-fighting  capa¬ 
bilities,  our  pilots’  success  in  combat  will  continue  to  rely  on  the 
dependable  operation  of  the  full  designed  performance  of  all  their 
avionics  equipment. 

The  Air  Force  s  challenge  will  also  mount  because  future  environ¬ 
ments  will  probably  impose  stricter  and  stricter  requirements  and  con¬ 
straints  on  warfare.  The  Air  Force  may  well  need  to  deploy  its  combat 
airplanes  over  long  distances  with  little  or  no  warning.  It  will  need 
combat  airplanes  in  sufficient  quantities  to  fight  at  times  and  places 
largely  determined  by  our  enemies.  Therefore,  the  Air  Force’s  air¬ 
planes  will  require  great  mobility  and  basing  flexibility.  Because  its 
limited  number  of  overseas  bases  will  probably  become  increasingly 
vulnerable  to  enemy  attack  and  rising  costs  will  preclude  preposition¬ 
ing  large  amounts  of  support  equipment,  the  Air  Force  will  be  forced  to 
rely  either  on  numerous  airlift  missions  or  on  decreased  support  for  its 
airplanes.  These  bases,  in  addition,  may  accommodate  far  fewer  air¬ 
planes  than  an  enemy  might  launch  against  them. 


PROBLEMS 

Our  research  has  found  that  problems  in  supporting  contemporary 
avionics  currently  pose  the  most  serious  threat  to  dependable  delivery 
of  full  designed  capabilities.  Moreover,  these  problems  stem  from  the 
weaknesses  that  Sec.  II  identifies  in  both  the  acquisition  and  the  sup¬ 
port  processes  for  avionics.  These  findings  are  drawn  from  an  assess¬ 
ment  based  upon  specially  collected  data. 

Data  Source 

The  source  is  the  special  data  collection  effort  conducted  during  the 
assessment  stage  of  the  F-15/F-16  Radar  R&M  Improvement  Program. 
That  was  an  exploratory  application  of  the  assessment  stage  for  the 
maturational  development  concept  described  in  Sec.  III. 

Data  on  the  F-16  A/B  radar  were  collected  from  150  F-16s  moni¬ 
tored  during  a  six-month  period  (June  December  1984)  at  Hill  AFB 
and  Hahn  Air  Base.  These  data  covered  16,077  flights  and  the  result¬ 
ing  radar  maintenance  at  these  bases.  Data  were  acquired  by  contrac- 


'See  U.S.  Department  of  Defense,  1985;  Rich,  Stanley,  and  Anderson,  1984;  Rich  and 
Dews,  1986. 


tor  personnel2  who  interviewed  pilots  after  maintenance  debriefing  and 
documented  all  flight  anomalies  observed  by  the  pilot,  including  faults 
detected  by  built-in  test  (BIT).  Contractor  personnel  also  documented 
maintenance  on  the  flightline,  in  the  shop,  and  at  the  depot. 

Similar  data  on  the  F-15  C/D  radar  were  collected  from  150  F-15s 
monitored  during  the  same  six-month  period  at  Langley  AFB  and  Bit- 
burg  Air  Base.  These  data  covered  16,702  flights  and  the  resulting 
radar  maintenance  at  these  bases.3 

An  Assessment 

Drawing  on  information  from  these  special  data  collection  efforts, 
Sec.  IV  shows  that  the  main  problems  of  supporting  modern  avionics 
are  the  difficulties  of  detecting  and  correcting  faults  in  equipment. 
Various  factors  contribute  to  such  difficulties: 

•  Multiple  operating  modes  and  environments. 

•  Limited  opportunities  to  exercise  certain  subsystems. 

•  Rare  occurrences  of  total  failures. 

Multiple  Operating  Modes  and  Environments.  Sophisticated  avionics 
equipment  (such  as  fire  control  radars)  operate  in  several  modes  and 
environments  that  can  influence  whether  symptoms  of  a  fault  are 
observable: 

•  Some  problems  manifest  symptoms  in  one  operating  mode  but 
not  in  another.  The  pilot  directly  controls  some  radar  operating 
modes  by  activating  switches  on  the  control  panel,  the  throttle, 
and  the  stick;  computers  directly  control  other  modes.  In  addi¬ 
tion,  the  pilot  often  rapidly  executes  a  series  of  modes  (e.g., 
search,  acquire,  track,  weapon  release)  while  coping  with  a  high 
workload  and,  in  the  case  of  fighter  pilots,  a  physically  exhaust¬ 
ing  series  of  maneuvers.  Thus  when  a  problem  arises,  the  pilot 
may  not  know  the  operating  mode.  And  even  if  he  does  know  the 
operating  mode,  he  probably  cannot  subsequently  identify  the 
precise  settings  of  all  switches  at  the  time  of  the  problem  for  the 
benefit  of  maintenance  personnel. 

•  Some  problems  occur  in  one  operating  environment  but  not  in 
another.  For  example,  subsystems  that  rely  on  sensors  (such  as 

2The  data  collection  team  included  32  people  provided  by  the  radar  contractor  (We9t- 
inghouse),  and  four  people  provided  by  the  weapon  system  prime  contractor  (General 
Dynamics). 

3This  effort  required  the  fielding  of  a  data  collection  team  that  included  36  people,  32 
provided  by  the  radar  contractor  (Hughes)  and  four  provided  by  the  weapon  system 
prime  contractor  (McDonnell  Aircraft). 
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the  antenna  in  a  radar)  for  their  primary  source  of  information 
perform  more  or  less  well  depending  on  the  environment  in 
which  the  object  is  sensed  and  on  the  object’s  movement  within 
that  environment.  In  addition,  some  problems  (such  as  mechani¬ 
cal  flaws  in  connections)  are  triggered  by  environmental  changes 
(temperature,  vibration,  and  deflection  under  high  flight  loads — 
e.g.,  pulling  nine  gs).  High  speed  flight  at  low  altitude  especially 
stresses  electronics  because  of  the  vibrations  and  high  tempera¬ 
tures  that  accompany  it.  When  a  problem  arises,  the  pilot  may 
not  know  or  remember  all  environmental  conditions  that  influ¬ 
ence  the  operation  of  his  equipment. 

Faults  in  sophisticated  avionics  subsystems  often  exhibit  symptoms 
that  depend  on  the  operating  mode  and  environment.  In  some  operat¬ 
ing  modes  and  environments,  the  equipment  may  even  function  prop¬ 
erly;  in  others,  it  may  deliver  only  part  of  its  designed  performance;  in 
yet  others,  it  may  fail  completely. 

Limited  Opportunities  to  Exercise  Certain  Subsystems.  Routine 
peacetime  missions  provide  limited  opportunities  to  fully  exercise  cer¬ 
tain  combat-oriented  subsystems,  such  as  fire  control  radars,  air-to-air 
weapon  delivery  systems,  and  electronic  warfare  (EW)  systems.  Fur¬ 
thermore,  routine  peacetime  missions  may  tolerate  degraded  capabili¬ 
ties  that  could  be  detrimental  in  combat.  Thus  maintenance  personnel 
and  contractors  can  rarely  discern  the  full  extent  and  effect  of  faults. 

Rare  Occurrences  of  Total  Failures.  Perhaps  most  important,  when 
modern  avionics  fail  to  deliver  their  full  measure  of  designed  capability, 
the  failure  is  usually  manifested  in  terms  of  a  performance  degradation 
for  one  or  more  functions  rather  than  a  total  loss  of  those  functions. 
There  are  two  reasons  for  these  failures: 

1.  Because  of  the  broad  versatility  of  avionics  equipment,  usually 
some  functions  will  work  well  and  other  functions  will  produce 
degraded  levels  of  performance.  Although  a  degraded  condition 
may  seem  tolerable  during  peacetime  exercises,  during  combat 
airplanes  must  deliver  everything  they  were  designed  to  do. 

2.  The  degraded  or  failed  performance  of  avionics  equipment  often 
generates  intermittent  symptoms,  because  pilots  do  not 

—  Use  identical  combinations  of  avionics  equipment  on  all 
flights, 

—  Use  identical  modes  of  operation  on  all  flights, 

—  Expose  avionics  equipment  to  identical  environmental  and 
stress  conditions  on  all  flights. 
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In  light  of  the  problems  of  intermittent  observability  of  degraded 
performance,  maintenance  technicians  and  R&M  managers  need  to 
make  a  formal  distinction  between  what  we  term  Type  A  and  Type  B 
faults: 

•  Type  A  faults  have  what  we  call  stationary  observability.  The 
processes  used  to  observe  their  symptoms  are  inherently  sta¬ 
tionary,  ah  ays  yielding  the  same  results.  Type  A  faults  manifest 
symptoms  that  are  observable  no  matter  when  or  where  the 
equipment  is  operated  or  tested.  A  broken  picture  tube  in  a  video 
display  is  a  Type  A  fault. 

•  Type  B  faults  have  what  we  call  nonstationary  observability. 
They  only  manifest  symptoms  some  of  the  time.  A  loosely  sol¬ 
dered  wire  can  be  a  Type  B  fault.  It  may  noticeably  deteriorate  a 
video  display  depending  on  how  the  radar  is  being  used  (the 
specific  functional  mode)  or  on  how  much  it  is  jostled  (the 
specific  environments).  Because  of  this  now-you-see-it-now-you- 
don’t  quality,  pilots  and  maintenance  personnel  often  question 
whether  specific  Type  B  faults  actually  exist. 

Because  Type  B  faults  are  often  difficult  to  define,  measure,  and  inter¬ 
pret,  currently  used  performance  measures  tend  to  presume  that  all 
failures  of  avionics  equipment  are  Type  A  faults.  But  special  data  col¬ 
lection  and  analysis  efforts  have  determined  that  the  major  weakness 
of  avionics  R&M  is  Type  B  faults:  They  are  hard  to  locate  and  to 
eliminate.  They  pose  major  problems  for  the  current  support  process. 


MEETING  THE  CHALLENGES  AND  ADDRESSING 
THE  PROBLEMS 

The  challenge  before  the  Air  Force  is  to  provide  high  performance 
avionics  that  can  dependably  deliver  the  full  measure  of  designed  capa¬ 
bilities.  To  meet  this  challenge,  the  Air  Force  must  address  the  serious 
support  problems  posed  by  avionics  faults  in  general  and  Type  B  faults 
in  particular.  To  this  end  we  offer  a  six-part  strategy  for  reforming  the 
avionics  acquisition  and  support  processes.  The  first  proposal 
addresses  major  weaknesses  in  the  acquisition  process  contributing  to 
support  problems: 

1.  Accelerate  R&M  Related  Avionics  Technologies.  The  Air 
Force  needs  to  speed  development  and  maturation  of  technolo¬ 
gies  that  promise  to  improve  the  R&M  of  avionics  equipment. 
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The  next  three  proposals  address  major  weaknesses  in  both  the 
acquisition  and  support  processes. 

2.  Improve  the  Ability  to  Test  Avionics  Equipment.  In  the 
airplane,  on  the  flightline,  in  the  shop,  and  at  the  depot,  the  Air 
Force  needs  to  ensure  that  future  development  efforts  concen¬ 
trate  on  isolating  faults  that  degrade  the  performance  of  avionics 
equipment. 

3.  Provide  More  Complete  Feedback  on  Equipment  Perfor¬ 
mance.  The  Air  Force  needs  to  improve  the  quality  of  informa¬ 
tion  received  from  the  pilot  debrief  and  institute  serial  number 
tracking  of  equipment  to  assist  maintenance  personnel  in  restor¬ 
ing  subsystems  to  their  full  complement  of  designed  capability. 
This  information  will  also  provide  data  for  product  improve¬ 
ment. 

4.  Adopt  a  Maintainability  Indicator.  The  Air  Force  needs  to 
adopt  a  maintainability  indicator  that  measures  the  efficiency  of 
the  support  process  in  removing  and  fixing  faulty  avionics  equip¬ 
ment.  It  would  be  used  in  conjunction  with  reliability  indicators 
to  determine  whether  product  improvement  is  needed. 

The  fifth  proposal  addresses  a  major  weakness  in  the  acquisition  pro¬ 
cess,  whose  persistence  is  occasioned  by  insufficient  feedback  from  the 
support  process. 

5.  Institute  Maturational  Development.  For  selected  combat- 
essential  avionics  subsystems,  the  Air  Force  needs  to  institute 
maturational  development  that  is  a  multi-cycle  development  pro¬ 
cess,  including  operational  tests,  aimed  exclusively  at  identifying 
and  correcting  fault-isolation  deficiencies,  failure  modes,  and 
their  frequencies  of  occurrence  in  selected  combat-essential  and 
technologically  sophisticated  avionics  subsystems. 

The  final  proposal  addresses  a  major  weakness  that  limits  the  Air 
Force’s  ability  to  implement  the  foregoing  proposals.  The  weakness 
stems  from  problems  with  the  current  organization  of  the  Air  Force’s 
avionics  engineering  resources. 

6.  Reorganize  Avionics  Engineering  Resources.  To  more 
closely  oversee  the  research,  development,  and  maturation  of 
avionics  equipment,  the  Air  Force  needs  to  take  a  fresh  look  at 
how  avionics  engineering  management  resources  are  organized. 
It  should  examine  the  alternative  of  instituting  an  Avionics 
Engineering  Center  assisted  by  contractor  organizations. 
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In  sum,  this  report  recommends  that  sophisticated  combat-essential 
avionics  subsystems  can  benefit  from  increased  special  attention  to 
ensure  that  they  are  reliable  and  easily  maintained.  Without  a  fully 
reliable  engine,  an  airplane  (especially  a  single-engine  airplane)  cannot 
safely  fly;  without  a  fully  reliable  combat  avionics  suite,  a  combat  air¬ 
plane  cannot  survive  in  a  high-threat  environment.  Moreover,  the 
sophistication  of  this  avionics  equipment  is  essential  for  these  airplanes 
to  carry  out  their  combat  missions.  To  tolerate  less  than  fully  mature 
equipment  reduces  the  unique  and  necessary  capabilities  of  these  air¬ 
planes.  In  addition,  to  assure  timely  and  fully  beneficial  execution  of  a 
maturational  development  phase,  this  report  recommends  that  the  Air 
Force  formally  incorporate  this  approach  as  an  explicit  phase  within 
the  acquisition  process. 


ORGANIZATION  OF  THIS  DOCUMENT 

Section  II  describes  weaknesses  in  the  acquisition  and  support 
processes  for  modern  avionics  that  lead  to  the  problems  the  Air  Force 
encounters  in  supporting  modern  avionics.  Because  instituting  matura¬ 
tional  development  (Proposal  5)  is  a  key  part  of  the  strategy  for 
reforming  these  processes,  Sec.  Ill  describes  the  concept  and  Sec.  IV 
describes  exploratory  applications  of  maturational  development  to  the 
F-15  C/D  and  F-16  A/B  fire  control  radars.  Section  V  suggests  ways 
of  strengthening  Air  Force  initiatives  that  are  expected  to  improve 
avionics  R&M  and  make  maturational  development  more  cost  effec¬ 
tive.4  Section  VI  identifies  further  initiatives  aimed  at  making  matura¬ 
tional  development  even  more  cost  effective.  The  findings  from  Secs. 
V  and  VI  form  the  basis  for  Proposals  I  through  4,  and  the  findings 
from  Secs.  IV  through  VI  form  the  basis  for  Proposals  5  and  6.  Sec¬ 
tion  VII  summarizes  the  recommendations  and  presents  conclusions. 
Appendix  A  contains  a  preliminary  assessment  of  avionics  maturity 
conducted  by  RAND  in  1980,  and  App.  B  describes  how  this  prelim¬ 
inary  assessment  led  to  the  exploratory  applications  of  maturational 
development  described  in  Sec.  IV. 


4Such  initiatives  are  sometimes  viewed  as  competing  alternatives  to  maturational 
development.  The  strategy  put  forth  here  views  them  as  important  complements. 


H.  WEAKNESSES  IN  THE  ACQUISITION 
AND  SUPPORT  PROCESSES  FOR 
MODERN  AVIONICS 


THE  ACQUISITION  PROCESS 

In  acquiring  modem  fighter  airplanes,  the  Air  Force  must  balance 
conflicting  goals.  To  acquire  adequate  numbers  of  airplanes,  it  must 
hold  acquisition  costs  to  a  minimum,  which  in  turn  also  means  holding 
aircraft  size  and  weight  to  a  minimum.  However,  the  Air  Force  must 
also  build  into  its  airplanes  avionics  equipment  necessary  to  ensure 
effective  performance  of  combat  missions.  And  such  equipment  inevi¬ 
tably  adds  to  the  cost,  size,  and  weight  of  airplanes. 

To  balance  these  goals,  the  Air  Force  has  increasingly  relied  on 
advancing  technologies  and  highly  integrated  suites  of  avionics  subsys¬ 
tems  to  avoid  sacrificing  improved  functional  performance  for  the  sake 
of  cost,  size,  and  weight.  These  advancing  technologies  and  integrated 
subsystems  have  clearly  improved  functional  performance.  They  have 
also  led  to  miniaturization  and  multifunctional  use  of  avionics  com¬ 
ponents,  and  increased  interdependence  among  avionics  subsystems 
(see  Fig.  2.1). 

As  General  Lawrence  A.  Skantze,  Commander  of  the  Air  Force  Sys¬ 
tems  Command,  has  pointed  out,  our  principle  has  been  “to  take  some 
risks,  but  on  a  prudent  basis”  when  developing  and  applying  advancing 
technologies,  and  we  must  therefore  “expect  some  technical  problems, 
but  ones  we  can  solve.”1  The  Air  Force  works  hard  to  achieve  the  per¬ 
formance  edge,  and  it  admittedly  takes  risks  to  do  so. 

Among  the  technical  problems  we  must  expect  is  that  avionics 
equipment  sometimes  does  not  regularly  perform  at  all  of  its  designed 
capability.2  That  inability  results  from  weaknesses  in  the  ways  the  Air 

'Meyer  and  Schemmer,  1985,  p.  64. 

2For  one  example,  the  special  data  collection  effort  of  the  Radar  R&M  Improvement 
Program  found  a  problem  with  the  radar  in  F-16  A  number  0752  that  indicated  pro¬ 
longed  degraded  performance.  Three  days  after  that  radar’s  Low  Power  Radio  Frequency 
(LPRF)  unit  was  replaced  with  a  new  one,  the  pilot  experienced  lock-on  problems  with 
the  radar.  Four  days  later,  the  pilot  reported  the  radar  could  not  pick  up  a  tanker  until 
it  was  25  miles  away;  a  properly  functioning  radar  should  have  picked  up  the  tanker  at 
more  than  double  that  distance.  This  same  radar  experienced  problems  indicating 
degraded  performance  for  the  next  five  months.  Section  IV  presents  additional  examples 
where  persistent  problems  influence  the  deliverable  performance  of  sophisticated  avionics 
equipment. 
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Note:  Numbers  refer  to  data  word  types 

Fig.  2.1— Complex  interrelations  among  avionics  subsystems:  The  number 
of  different  data  words  transmitted  between  critical 
subsystems  for  the  F-16  A/B 

Force  acquires  avionics  equipment.  Many  problems  that  arise  in  sup¬ 
porting  modern  avionics  are  not  foreseen  during  the  development  pro¬ 
cess  and  are  not  adequately  addressed  during  post-development 
maturation  because  the  acquisition  process  is  not  structured  to  deal 
effectively  with  such  problems. 

The  Challenges  of  Foreseeing  Problems  During  Development 

The  factors  that  make  many  support  problems  hard  to  foresee  dur¬ 
ing  development  can  be  grouped  in  three  categories: 

•  Equipment  design 

•  Equipment  test 

•  Integration  of  contractor  efforts 

Equipment  Design.  The  avionics  designer  must  take  developing 
technologies  and  incorporate  them  into  a  large  and  highly  integrated 
subsystem.  He  must  squeeze  the  weight  and  size  of  this  subsystem  into 
a  fighter  airplane  where  it  will  be  subjected  to  extreme  temperatures 
and  stresses  as  it  is  buffeted  about  the  sky. 
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Equipment  Test.  The  developer  usually  tests  his  subsystem  on  the 
ground  in  a  benign  environment  using  tests  and  pass/fail  criteria  that 
differ  from  those  ultimately  used  in  the  airplane,  in  the  airbase’s  avion¬ 
ics  shop,  and  at  the  depot.  All  these  tests  differ  not  only  because  dif¬ 
ferent  companies  build  the  test  equipment  for  each  maintenance  level 
but  also  because  the  tests  must  become  more  detailed  as  the  subsystem 
is  disassembled  at  each  maintenance  level.  These  differences  make  it 
hard  to  demonstrate  that  a  subsystem  will  pass  tests  when  it  is 
reassembled  and  reintroduced  to  the  flight  environment. 

Integration  of  Contractor  Efforts.  Many  companies  are  involved 
in  development  efforts.  One  company  builds  i,ne  airframe,  another 
builds  a  specific  avionics  subsystem,  others  build  subsystems  that  will 
interface  with  the  specific  avionics  subsystem,  and  yet  others  build  test 
equipment  for  the  specific  avionics  subsystem.  The  challenge  of  coor¬ 
dinating  such  efforts  is  greatly  complicated  by  the  complex  interfaces 
required  to  mechanize  a  highly  integrated  suite  of  avionics  subsystems. 

The  Weaknesses  in  Addressing  the  Challenges 

The  acquisition  process  has  major  weaknesses  when  it  comes  to 
addressing  hard-to-foresee  support  problems.  The  most  serious  fall 
into  five  categories: 

•  Engineering  data  on  new  technologies 

•  Corporate  memory 

•  Time  available  for  development  and  maturation 

•  Information  on  support  problems  with  new  equipment 

•  Transfer  of  program  (including  engineering)  management 
responsibility. 

Lack  of  Engineering  Data  on  New  Technologies.  Because 
avionics  equipment  so  often  relies  on  the  “cutting  edge”  of  technology, 
technological  developments  can  move  rapidly  from  the  experimental 
stage  to  application  in  a  full-scale  engineering  development  program. 
Such  rapid  developments  limit  the  time  that  companies  have  to  collect 
engineering  data  to  support  the  design  process.  Moreover,  competitive 
pressures  make  companies  reluctant  to  share  engineering  data  on  the 
most  promising  new  technologies.  Consequently,  engineering  databases 
for  such  technologies  can  grow  very  slowly.  Slow  accumulation  of 
information  on  failure  modes  and  effects  contributes  to  designs  with 
hard-to-foresee  support  problems. 

Insufficient  Corporate  Memory.  Failure  to  accumulate  and 
disseminate  information  on  hard-to-foresee  problems  that  reach  the 
field  is  another  weakness  that  contributes  to  current  support  problems. 
The  Air  Force  lacks  a  sound  vehicle  for  building  corporate  memory,  in 
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part  because  it  typically  relies  on  a  weapon  system  prime  contractor  to 
manage  the  development  and  initial  support  for  avionics  subsystems. 

Inadequate  Time  to  Develop  and  Mature  Equipment.  Avionics 
subsystems  have  usually  started  full-scale  engineering  development 
(FSED)  last,  even  though  they  are  probably  the  most  complex  areas  of 
the  airplane  and  would  benefit  greatly  from  additional  development 
time.  The  engine,  another  complex  subsystem,  typically  starts  FSED 
many  years  ahead  of  the  combat  avionics.3  And  the  airframe  typically 
starts  FSED  anywhere  from  six  months  to  a  year  before  the  airframe 
contractor  signs  FSED  contracts  with  the  avionics  subcontractors. 

Figure  2.2  shows  the  major  milestones  in  the  acquisition  and 
development  of  the  F-15.  Concept  formulation  for  the  F-15  began  in 
June  1967,  and  FSED  for  the  airframe  began  in  January  1970. 
Although  initial  development  for  the  engine  began  roughly  16  months 
before  FSED  for  the  airframe,  FSED  for  the  avionics  systems  began 
considerably  later.  FSED  for  the  inertial  navigation  system  (INS) 
began  roughly  eight  months  later,  and  FSED  for  the  fire  control  radar 
system  began  roughly  11  months  later. 

Lack  of  Information  on  Support  Problems.  Not  only  do  avion¬ 
ics  developers  typically  lack  adequate  time  to  mature  their  equipment 
before  it  is  delivered  to  operational  squadrons,  but  shortly  after  fielding 
the  initial  squadron,  developers  lose  touch  with  many  of  the  opera¬ 
tional  realities  in  which  their  designs  must  survive.  This  means,  for 
instance,  that  they  lack  engineering  data  on  how  temperatures,  vibra¬ 
tions,  and  flight  loads  may  be  punishing  the  more  fragile  parts  of  their 
design.  Flight  testing  does  occur  during  Developmental  Test  and 
Evaluation  (DT&E)  and  Operational  Test  and  Evaluation  (OT&E) 
phases,  but  such  testing  has  not  provided  the  kind  of  engineering 
assessments  that  are  needed  to  address  all  of  the  R&M  weaknesses.4 

3Recognizing  the  sophisticated  and  flight-essential  nature  of  turbine  engines,  the  Air 
Force  usually  (1)  contracts  directly  for  their  development;  (2)  starts  this  development 
long  before  that  of  the  airframe;  and  (3)  precedes  this  with  many  years  of  development, 
testing,  and  redevelopment  of  the  engine’s  more  critical  components. 

4Although  many  R&M  problems  are  identified  and  addressed  as  a  result  of  DT&E 
and  OT&E,  many  others  are  not.  DT&E  occurs  early,  and  involves  a  considerable 
engineering  presence,  but  the  avionics  equipment  often  is  still  undergoing  major  modifi¬ 
cations,  the  ground  test  equipment  has  not  completed  development,  and  the  database 
(number  of  flights)  is  very  small.  Because  OT&E  occurs  later,  the  configuration  of  the 
avionics  equipment  is  more  stable,  some  ground  test  equipment  may  be  available,  and  the 
database  is  larger.  However,  because  testing  concentrates  on  operational  evaluation 
rather  than  engineering  analysis,  the  engineering  involvement  is  greatly  reduced  from 
DT&E.  Consequently,  the  resulting  database  often  reveals  only  symptoms  of  R&M  prob¬ 
lems,  and  not  the  root  causes,  leading  to  ambiguities  about  whether  design  changes 
already  in  progress  may  resolve  root  causes.  Program  managers  find  themselves  fre¬ 
quently  tempted  to  take  positions  that  the  root  causes  are  already  being  addressed.  As 
Sec.  IV  demonstrates,  too  often  they  are  not. 
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Further  data  collection  and  engineering  analysis  based  on  early  opera¬ 
tional  experiences  are  required  to  build  a  suitably  large  database  that 
reflects  operational  R&M  problems,  including  the  interaction  between 
the  airborne  equipment  and  the  ground-based  test  equipment. 

Problematic  Timing  of  Transfer  of  Program  Management 
Responsibility.  In  the  case  of  avionics,  engineering  management 
responsibilities  are  transferred  from  the  weapon  system  System  Pro¬ 
gram  Office  (SPO)  to  the  Air  Force  Logistics  Command  (AFLC)  long 
before  the  airborne  equipment  and  its  support  equipment  have  matured 
to  appropriate  levels  of  R&M.5  This  transfer  of  engineering  responsibil¬ 
ity  generally  occurs  during  the  equipment’s  early  operational  life  when 
engineers  assigned  to  the  SPO  are  only  beginning  to  have  an  opportu¬ 
nity  to  identify  the  equipment’s  strengths  and  weaknesses.  Once  the 
AFLC  System  Program  Manager  assumes  responsibility,  a  new — and 
much  smaller — group  of  Air  Force  engineers  becomes  responsible  for 
the  equipment’s  further  maturation.  These  engineers  must  also  initiate 
entirely  new  contracting  documents  to  begin  any  redesign  work.  This 
is  in  contrast  to  the  far  simpler  contracting  procedure  that  the  SPO 
follows  to  accomplish  the  same  sort  of  design  changes  before  Program 
Management  Responsibility  Transfer  (PMRT).  Thus,  any  improve¬ 
ments  in  equipment  maturation  slow  considerably  after  PMRT.  With 
an  impending  transfer  of  R&M  engineering  responsibilities,  it  is  also 
difficult  to  keep  the  development  organization  interested  in  improving 
R&M. 

Consequences 

As  a  consequence  of  weaknesses  in  the  acquisition  process,  the  Air 
Force  must  often  be  satisfied  with  airborne  equipment  and  associated 
support  equipment  that  has  not  matured.  R&M  has  not  been 
developed  to  the  point  that  the  equipment  can  regularly  perform  at  the 
highest  level  of  its  designed  capability  given  available  support 
resources.  Less  mature  avionics  equipment  requires  larger  amounts  of 
support;  more  mature  avionics  equipment  requires  smaller  amounts  of 
support.  This  suggests  a  simple  tradeoff:  The  Air  Force  ought  to  be 
able  to  compensate  for  less  mature  avionics  equipment  merely  by  sup¬ 
plying  additional  maintenance  personnel,  maintenance  equipment, 
spare  parts,  and  the  like. 

5The  appropriate  level  of  R&M  maturity  will  vary  by  subsystem  and  will  depend  upon 
current  R&M  circumstances  of  each  development  program.  In  each  case,  the  probable 
value  of  a  marginal  increase  in  R&M  maturity  must  be  analyzed  in  light  of  the  probable 
marginal  cost.  Identification  of  the  most  worthwhile  opportunities  requires  a  sound 
engineering  analysis  for  both  the  current  R&M  situation  and  competing  alternatives  for 
improvement.  Unfortunately,  current  testing  (both  DT&E  and  OT&E)  often  does  not 
provide  an  adequate  basis  for  such  analyses. 
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Such  a  tradeoff  might  seem  sensible  during  peacetime,  especially 
when  one  considers  that  the  maturing  of  avionics  equipment  requires 
the  up-front  expenditure  of  additional  development  time  and  money. 
However,  large  amounts  of  support  are  extremely  expensive  and  can 
never  compensate  for  lower  levels  of  R&M  maturity  during  wartime. 
Indeed,  heavy  wartime  reliance  on  support  decreases  mobility  of  forces, 
exposes  large  amounts  of  equipment  and  personnel  to  enemy  action, 
and  places  demands  on  scarce  transportation  resources;  even  additional 
support  often  cannot  assure  that  the  full  amount  of  design  capability 
has  been  restored.6  In  sum,  heavy  wartime  reliance  on  support  rather 
than  on  more  mature  R&M  for  avionics  equipment  unfortunately  leads 
to  lower  performance  than  the  original  design  of  the  weapon  system 
sought. 

To  compound  matters,  even  with  reliance  on  large  amounts  of  sup¬ 
port  equipment,  weaknesses  in  the  support  process  hinder  the  Air 
Force’s  ability  to  support  less  than  fully  mature  avionics. 


THE  SUPPORT  PROCESS 

In  supporting  modern  fighter  airplanes,  the  Air  Force  must  balance 
conflicting  goals.  To  have  adequate  numbers  of  pilots  trained  for  com¬ 
bat,  it  must  sustain  a  certain  level  of  training  during  peacetime.  Like¬ 
wise,  to  have  adequate  numbers  of  airplanes  operationally  ready  for 
immediate  commitment  to  combat  during  peacetime  but  especially  dur¬ 
ing  war,  it  must  quickly  complete  maintenance  actions.  Rapid  sortie 
generation  requires  trained  maintenance  technicians  to  rapidly  com¬ 
plete  maintenance  actions.  Thus,  peacetime  training  needs  for  both 
pilots  and  maintenance  technicians  demand  quick  response  by  the  sup¬ 
port  process.  However,  the  Air  Force  must  also  take  time  to  document 
support  problems  and  track  the  adequacy  of  the  equipment’s  perfor¬ 
mance. 

Among  the  support  problems  we  must  expect  is  that  specific  items  of 
airborne  equipment  will  not  regularly  perform  at  the  level  of  their 
designed  capability  and  also  that  pilots  sometimes  do  not  report  their 
observations  of  degraded  performance.  Neither  the  acquisition  process 
nor  the  support  process  adequately  addresses  the  difficult  support  prob¬ 
lems  that  arise  in  supporting  modern  avionics. 


6Section  IV  shows  how  difficult  it  can  be  to  restore  the  full  amount  of  designed  capa¬ 
bilities  when  R&M  maturity  is  lacking. 
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The  Challenges  Presented  by  Difficult  Support  Problems 

One  of  the  most  difficult  support  problems  is  isolating  a  Type  B 
fault,  one  that  evades  detection  at  one  or  more  places  in  the  support 
process.  The  avionics  support  processes  for  both  the  F-15  and  the  F-16 
rely  on  large  amounts  of  resources  to  compensate  for  the  current  matu¬ 
rity  level  of  avionics  equipment.  Support  for  this  equipment  resides  at 
large  operating  bases  with  complex  and  expensive  diagnostic,  support, 
and  repair  equipment.  There  are  large  supplies  of  spare  parts  and 
there  are  sophisticated  facilitir  •  for  repairing  support  materials. 

Notwithstanding  large  investments  in  such  materials,  the  support 
process  for  the  F-15  and  F-16  has  four  stages  (see  Fig.  2.3),  during 
which  the  symptoms  of  a  fault  can  evade  detection: 

•  The  cockpit 

•  The  flightline 

•  The  Avionics  Intermediate  Shop 

•  The  depot. 

The  Cockpit.  In  the  cockpit  the  pilot  or  a  maintenance  technician 
first  observes  symptoms  of  an  equipment  fault.  Here  a  fault  has  its 
first  opportunity  to  evade  detection.  Symptoms  may  be  in  the  form  of 
a  degradation  in  performance  detected  by  the  pilot  or  indicated  by  the 
built-in  test  (BIT). 

To  help  compensate  for  a  pilot’s  limited  ability  to  detect  faults  in 
sophisticated  avionics  equipment,  airplanes  employ  BITs  that  periodi¬ 
cally  test  selected  aspects  of  the  avionics  subsystem’s  operation.  How¬ 
ever,  the  capability  of  contemporary  BITs  is  limited.  First,  the  time 
between  tests  depends  on  the  avionics  subsystem’s  design  and  the  func¬ 
tion  being  executed.  Some  functions  place  such  heavy  demands  on  the 
subsystem’s  computing  resources  that  virtually  no  time  is  available  to 
execute  the  BIT.  Second,  the  BIT  only  indicates  that  avionics  equip¬ 
ment  has  either  passed  or  failed  a  certain  test;  it  does  not  indicate  gra¬ 
dations  of  passing  or  failing.  Notwithstanding  its  limitations,  the  BIT 
often  detects  symptoms  of  faults  that  a  pilot  doesn’t  notice,  sometimes 
because  they  are  not  yet  serious  problems.  At  other  times,  however, 
the  BIT  is  able  to  detect  serious  problems  that  would  become  obvious 
to  a  pilot  only  during  combat. 

A  further  opportunity  for  a  fault  to  evade  detection  is  created  by 
pilots’  reporting  practices.  After  each  flight,  the  pilot  assigns  a  i.-erfor- 
mance  code  for  each  of  the  aircraft’s  major  subsystems.  If  the  pilot 
gives  a  subsystem  a  Code  1,  maintenance  personnel  assume  that  the 
subsystem  is  performing  satisfactorily  for  its  next  mission.  Either  a 
Code  2  or  a  Code  3  is  a  signal  that  the  subsystem  is  not  performing 
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satisfactorily.  In  the  event  of  a  Code  2,  maintenance  personnel  will 
work  on  the  subsystem  when  it  is  convenient  to  do  so.  One  or  more 
flights  may  be  flown  in  the  meantime.  In  the  event  of  a  Code  3, 
maintenance  personnel  will  try  to  fix  the  subsystem  before  the 
airplane’s  next  flight. 

The  Flightline.  A  fault’s  next  opportunity  to  evade  detection 
occurs  during  flightline  maintenance.  If  dispatched  to  the  flightline, 
the  avionics  specialists  will  attempt  to  stimulate  symptoms  or,  more 
often,  try  to  repeat  the  BIT  indication,  often  difficult  to  do  in  the 
ground  environment.  If  they  can  stimulate  the  symptoms  or  the  BIT 
indications,  the  specialists  will  remove  the  line -replaceable  unit  (LRU) 
apparently  causing  the  symptoms;'  if  they  cannot,7 8  they  may  either 
send  the  aircraft  on  its  next  flight  or  decide  to  investigate  the  claim  of 
a  fault  further.  If  the  BIT  generates  the  reported  symptoms,  it  also 
produces  a  coded  message  that  sometimes  helps  the  flightline  special¬ 
ists  identify  the  faulty  equipment. 

The  Avionics  Intermediate  Shop.  Once  specialists  remove  an 
LRU  from  the  airplane,  they  replace  it  if  possible  with  a  spare  from 
supply  and  send  the  suspected  faulty  LRU  to  an  avionics  intermediate 
shop  (AIS)  for  detailed  performance  and  diagnostic  testing.9  The  next 
Opportunity  for  faults  to  evade  detection  can  arise  from  either  limita¬ 
tions  in  the  shop’s  capability  or  problems  with  the  performance  of  its 
equipment. 

The  F-15  has  over  100  avionics  LRUs,  and  each  contains  a  large 
number  of  shop  replaceable  units  (SRUs),  such  as  circuit  cards.  The 
LRU/SRU  concept  using  the  AIS  was  developed  to  avoid  holding  up 
airplanes  for  lengthy  diagnostic  tests  and  repairs,  which  are  run  only 
on  LRUs  that  are  removed  from  an  airplane  and  sent  to  the  avionics 
shop.  Each  F-15  AIS  has  eight  types  of  test  stations:  four  manual  sta¬ 
tions,  which  test  and  repair  56  LRU  types;  three  automatic  stations, 
which  test  and  repair  44  LRU  types;  and  one  type  of  station  for  the 
tactical  electronic  warfare  system  (TEWS),  which  tests  and  repairs 
other  LRU  types. 


7With  modern  avionics  equipment,  LRU  replacement  is  the  main  type  of  corrective 
action  that  can  be  taken  on  the  flightline.  As  systems  age,  however,  wiring  and  connec¬ 
tor  problems  account  for  a  growing  percentage  of  the  maintenance  actions.  Even  so, 
LRU  replacement  still  remains  the  dominant  form  of  corrective  action. 

8Failure  to  isolate  can  be  caused  by  technical  as  well  as  human  factors.  Contributing 
technical  factors  can  be  deficiencies  in  BIT,  inadequate  ground  support  equipment,  and 
insufficient  maintenance  instructions  (Technical  Orders). 

9In  the  case  of  the  F-15,  the  AIS  is  so  large  it  requires  at  least  three  C-141s  to  deploy 
a  single  set  of  test  equipment  with  itB  associated  power  generators  and  air  conditioners. 
When  in  operation,  this  single  set  requires  4500  sq  ft  of  floor  space. 
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To  allow  one  type  of  test  station  to  test  many  types  of  LRUs,  the 
AIS  uses  interface  test  adaptors  for  each  LRU  type.  The  AIS  can  test 
a  given  type  of  LRU,  however,  on  only  one  type  of  test  station.  More¬ 
over,  a  test  station  can  test  only  one  type  of  LRU  at  a  time;  and  for  a 
given  LRU  type,  it  can  test  only  one  unit  at  a  time.  Whether  a  shop 
has  enough  stations  depends  on  how  many  LRUs  require  testing,  the 
testing  time  for  each  LRU,  and  the  availability  of  the  test  station. 

The  availability  of  test  stations  and  their  interface  equipment  is 
most  strongly  influenced  by  the  availability  of  parts  to  fix  the  test 
equipment,  which  for  the  F-15  contains  nearly  40,000  types  of  replace¬ 
able  parts  and  assemblies.  Although  each  part  is  reasonably  reliable,  a 
typical  shop  almost  always  needs  replacement  parts  for  some  test  sta¬ 
tions.  Shops  with  at  least  two  sets  of  test  equipment  can  usually  main¬ 
tain  a  capability  to  test  and  repair  most  LRUs,  but  there  are  times 
when  the  test  equipment  is  unable  to  deliver  the  full  amount  of  its 
designed  capability. 

In  a  sense,  the  shops’  test  equipment  is  similar  to  an  airplane.  The 
test  equipment  rarely  quits  operating.  Most  often,  it  develops  a  fault 
that  degrades  its  test  capability.  Usually  such  a  fault  manifests  symp¬ 
toms  that  are  detected  by  the  test  equipment’s  own  built-in  test.  To 
avoid  confusion  with  the  aircraft’s  BIT,  we  use  the  term  “test  equip¬ 
ment  self  test”  to  refer  to  such  tests.  Like  the  aircraft,  the  test  equip¬ 
ment  can  develop  Type  B  faults.  And  as  with  the  aircraft,  correction 
of  such  faults  can  be  a  very  time  consuming  and  frustrating  process. 
Moreover,  when  the  test  equipment  self  test  indicates  the  detection  of 
a  fault,  the  maintenance  technician  usually  has  no  other  direct  indica¬ 
tion  to  corroborate  the  validity  of  the  indication.  It  is  not  uncommon, 
therefore,  to  have  Type  B  faults  present  in  an  item  of  test  equipment 
while  maintenance  technicians  are  using  the  test  equipment  to  test  and 
repair  an  LRU.  Such  a  fault  in  test  equipment  can  create  an  opportu¬ 
nity  for  a  fault  in  an  LRU  to  evade  detection. 

Thus,  in  the  shop,  there  are  two  basic  ways  that  a  fault  can  evade 
the  attention  of  maintenance  technicians: 

•  Technicians  fail  to  stimulate  and  detect  symptoms  of  the  fault. 

•  Technicians  succeed  in  detecting  symptoms  of  the  fault  but  fail 
to  isolate  the  cause. 

Failure  to  detect  or  isolate  can  be  caused  by  technical  as  well  as  human 
factors.  Contributing  technical  factors  include: 

•  Test  equipment  (along  with  associated  software  and  Technical 
Orders)  not  designed  to  detect  or  isolate  the  fault. 

•  Test  equipment  not  performing  up  to  its  designed  capabilities. 
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Human  factors  can  include  the  training,  skill,  and  experience  of  the 
technician.  These  factors  can  be  especially  important  because  the 
technician  is  also  responsible  for  most  of  the  tests  and  repairs  directed 
at  the  test  equipment. 

Section  IV  shows  that  the  technical  factors  are  the  major  source  of 
difficulty  because  fault  detection  and  isolation  by  the  shop  is  based 
upon  tests  and  pass-or-fail  criteria  different  from  those  used  by  the 
BIT  on  the  airplane.  Consequently,  its  tests  can  find  discrepancies 
that  are  completely  unrelated  to  the  problem  encountered  on  the  air¬ 
plane.  Although  shop  technicians  may  believe  that  they  have  restored 
the  full  performance  for  and  item  of  equipment,  the  equipment  may 
still  possess  the  fault  detected  on  the  airplane. 

The  Depot.  If  a  shop  cannot  repair  an  LRU,  it  sends  that  item  to  a 
depot  for  further  testing.  The  depot  has  the  same  types  of  stations  as 
those  in  the  shop.  It  also  has  additional  equipment  that  can  test  and 
repair  SRUs  within  LRUs. 

Diagnosis  and  repair  of  SRUs  by  the  depot  is  complicated  by  the 
fact  that  its  test  stations,  like  those  in  the  shop,  use  measurements  dif¬ 
ferent  from  those  used  by  the  BIT  on  the  airplane.  In  addition,  its 
equipment  for  testing  SRUs  uses  measurements  different  from  those 
used  elsewhere  in  the  support  system. 

In  the  depot,  a  fault  can  evade  the  attention  of  maintenance  techni¬ 
cians  in  the  same  basic  ways  as  at  the  shop;  contributing  technical  and 
human  factors  are  also  the  same. 

Weaknesses  in  Addressing  the  Challenges 

From  the  cockpit  through  the  depot  major  challenges  confront  the 
capability  of  the  support  process  to  deal  with  difficult  faults.  More¬ 
over,  the  support  process  has  major  weaknesses  in  its  ability  to  address 
the  challenges  of  efficiently  detecting  Type  B  faults.  The  most  serious 
weaknesses  fall  into  five  categories: 

•  Information  provided  to  maintenance  concerning  inflight  symp¬ 
toms  of  degraded  performance. 

•  Identification  of  equipment  that  has  not  been  fully  restored  to 
deliver  its  designed  capabilities. 

•  Flow  of  information  between  maintenance  levels. 

•  Maintenance  capability  to  detect  and  isolate  Type  B  faults  in 
the  shop  and  in  the  depot. 

•  Procedure  for  reporting  support  deficiencies  and  possible  solu¬ 
tions. 
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Inadequate  Information  Provided  to  Maintenance.  Because 
there  are  so  many  opportunities  for  Type  B  faults  to  evade  detection 
and  isolation  during  the  maintenance  process,  pilots  must  inform 
maintenance  of  all  indications  of  potential  problems  with  equipment 
performance,  especially  after  the  completion  of  a  maintenance  action. 
The  pilot  needs  to  report  all  symptoms,  whether  pilot  observations  or 
BIT-detected  faults,  at  the  post-flight  maintenance  debriefing.  The 
lack  of  a  consistent  practice  in  this  regard  is  a  fundamental  weakness 
of  the  current  support  process.  Frequently,  maintenance  lacks  ade¬ 
quate  information  about  avionics  equipment  performance  during  rou¬ 
tine  training  flights.  Pilot-observed  indications  of  faults  often  go 
unreported  to  technicians  even  when  the  BIT  corroborates  degradation 
in  the  equipment’s  performance. 

Ineffective  Identification  of  Unrestored  Equipment.  Even 
when  a  pilot  fully  reports  all  inflight  indications  of  potential  problems, 
a  systematic  process  is  necessary  for  integrating  such  information  with 
information  from  the  maintenance  process  to  quickly  identify  equip¬ 
ment  that  the  maintenance  process  has  failed  to  restore  to  its  designed 
capability.  The  current  support  process  lacks  an  effective  way  to  track 
avionics  equipment  performance  by  serial  number.  Such  a  capability 
could  facilitate  the  identification  and  repair  of  faulty  units  that 
repeatedly  circulate  between  shops  and  airplanes.10 

Insufficient  Flow  of  Information  Between  Maintenance  Lev¬ 
els.  Even  with  full  reporting  by  pilots,  and  even  with  an  effective  way 
to  track  equipment  by  serial  number,  a  further  link  is  required  for  the 
fully  effective  flow  of  information  essential  for  quickly  identifying 
equipment  with  hard-to-fix  faults.  This  last  link  is  a  mechanism  for 
translating  test  results  from  one  level  to  another.  Currently,  mainte¬ 
nance  technicians  in  the  avionics  shop  cannot  use  information  from 
the  aircraft-level  BIT  to  either  guide  or  verify  their  own  tests.  Simi¬ 
larly,  maintenance  technicians  in  the  depot  cannot  use  information 
from  either  the  shop-level  tests  or  the  aircraft-level  BIT.  Such  com¬ 
munication  barriers  create  a  major  weakness  in  the  support  process. 
Shop  or  depot-level  technicians  can  find  and  correct  out-of-tolerance 
conditions  that  are  totally  unrelated  to  the  inflight  fault,  but  nothing 
may  be  done  to  correct  the  inflight  fault.  Shop  or  depot  technicians 
believe  that  their  work  has  fixed  the  equipment  when  in  fact  their 
actions  are  irrelevant  to  the  condition  that  caused  the  equipment  to  be 
placed  in  the  maintenance  process.11 

10Such  capability  is  needed  for  both  LRUs  and  SRUs. 

"For  the  F-15  radar  database  discussed  in  Sec.  IV,  20  percent  of  the  LRUs  sent  to 
the  shop  fell  into  this  category. 
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Low  Maintenance  Capability  for  Type  B  Faults.  Appropriate 
remediation  of  all  of  the  foregoing  weaknesses  would  only  improve  the 
ability  to  more  quickly  identify  specific  equipments  with  hard-to-fix 
faults.  To  actually  repair  the  equipment,  special  tests  and  even  addi¬ 
tional  test  resources  may  be  required.  Two  major  weaknesses  exist. 
First,  most  LRU  tests  that  are  conducted  with  automatic  test  equip¬ 
ment  have  limited  flexibility.  Technicians  cannot  break  into  the 
lengthy  test  sequence  to  repetitively  run  a  block  of  tests  in  an  area 
where  a  Type  B  fault  may  reside.  Moreover,  the  test  sequence  usually 
must  start  at  a  fixed  point.  The  unit  of  equipment  under  test  will 
therefore  achieve  a  thermal  equilibrium  long  before  most  of  the  tests 
are  executed,  making  it  difficult  to  find  thermally  sensitive  faults.  A 
second  weakness  is  the  lack  of  capability  to  control  the  environment 
for  the  unit  of  equipment  under  test.  Unfortunately,  many  Type  B 
faults  are  sensitive  to  flight-environment.  Environmental  chambers, 
while  not  practical  for  airbase-level  avionics  shops,  could  be  installed  at 
depot  test  facilities  to  provide  some  capability  to  represent  the  flight 
environment  (both  thermal  and  vibration  aspects). 

Inadequate  Procedures  for  Reporting  Support  Deficiencies. 
Finally,  it  is  reasonable  to  expect  deficiencies  in  the  support  process  for 
large,  complex  subsystems  that  rely  on  leading  edge  advances  in  tech¬ 
nology  for  much  of  their  important  capabilities.  Deficiencies  can  be 
expected  with  tests  at  all  levels  (BIT,  shop,  and  depot),  with  test 
equipment  (shop  and  depot)  maintenance,  and  with  maintenance 
instructions  (Technical  Orders)  at  all  levels.  But,  even  with  such 
expectations,  the  support  process  lacks  an  effective  and  efficient 
mechanism  for  gathering  feedback  from  maintenance  technicians. 

Consequences 

As  a  consequence  of  weaknesses  in  the  support  process,  the  Air 
Force  must  often  be  satisfied  with  equipment  that  is  delivering  less 
than  the  full  measure  of  what  it  was  designed  to  do.  Although  such 
equipment  usually  can  perform  mission-essential  functions,  the  ques¬ 
tion  is  how  well  would  the  host  weapon  system  fare  against  the  front¬ 
line  equipment  of  a  sophisticated  adversary?  A  radar  that  can  depend¬ 
ably  find  targets  at  its  full  design  range  is  clearly  preferable  to  one 
that,  although  still  functioning,  is  seriously  degraded  in  detection  capa¬ 
bility.  The  degree  of  functional  degradation  often  does  not  make  much 
difference  during  peacetime  training  flights,  but  it  can  make  a  crucial 
difference  in  combat.  Weaknesses  in  the  support  process,  therefore, 
pose  a  very  insidious  threat  to  sustaining  combat  superiority. 


III.  THE  CONCEPT  OF  MATURATIONAL 
DEVELOPMENT 


This  section  describes  the  observations  that  led  to  the  formulation 
of  the  concept  of  maturational  development.  It  then  defines  the  pro¬ 
posed  concept  and  suggests  some  areas  where  implementation  would  be 
most  beneficial.  Finally,  it  addresses  some  concerns  regarding  the 
concept’s  practicality. 


OBSERVATIONS 

Formulation  of  the  concept  of  maturational  development  was 
strongly  influenced  by  two  classes  of  observations.  The  first  is  a  set  of 
specific  observations  drawn  from  two  development  programs  from  the 
1960s.  The  second  is  a  set  of  general  observations  drawn  from  a  com¬ 
bination  of  those  experiences  from  the  1960s  and  twenty  years  of  sub¬ 
sequent  RAND  research  on  avionics  acquisition  and  support. 

Specific  Observations 

During  the  1960s,  two  development  programs  were  especially 
noteworthy  for  making  impressive  improvements  in  reliability  after  the 
initial  fielding  of  equipment: 

•  Minuteman  I  guidance  subsystem 

•  Carousel  inertial  navigation  subsystem 

Each  program  required  the  application  of  the  most  recent  advances  in 
technology  to  what  were,  at  the  time,  very  complex  subsystems. 
Although  these  subsystems  differ  from  modern  avionics  in  both  tech¬ 
nologies  and  complexity,  we  can  draw  important  lessons  from  the 
processes  used  to  improve  their  reliability. 

Minuteman  Guidance  Subsystem.  In  developing  the  missile 
guidance  subsystem  for  Minuteman  I,  the  Air  Force  aimed  at  achieving 
a  high  mean  time  between  removal  (MTBR).  When  it  went  into  initial 
operating  capability  (IOC),  Minuteman  I  had  an  MTBR  of  600  hours.1 
Although  this  would  be  good  for  an  avionics  LRU,  it  was  unacceptable 

'information  about  the  Minuteman  I  program  is  drawn  from  the  personal  files  of 
H.  L.  Shulman,  who  participated  in  the  Air  Force’s  reliability  growth  program  for  the 
guidance  subsystem. 
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for  the  Minuteman,  which  should  operate  continuously  (more  than 
8700  hours  per  year)  to  be  ready  for  immediate  firing.  With  this 
MTBR  of  600  hours,  the  Air  Force  had  to  remove  each  Minuteman’s 
inertial  guidance  equipment  from  the  silo  and  send  it  to  the  repair 
facility  at  Newark,  Ohio,  on  an  average  of  once  every  25  days.  Each 
time  this  occurred,  the  missile  was  out  of  service  for  roughly  seven  days 
because  of  the  time  required  to  remove  the  guidance  system  and  to 
replace  it  with  one  that  had  to  be  warmed  up  and  calibrated.  Even  if 
the  Air  Force  had  unlimited  spare  guidance  subsystems,  about  one- 
fourth  of  its  missiles  would  be  unavailable  at  any  given  time.  And 
spares  were  in  short  supply,  so  many  more  missiles  were  unavailable. 
This  extremely  serious  availability  problem  caused  the  Air  Force  to  ini¬ 
tiate  a  second  development  effort  aimed  exclusively  at  reliability 
improvement.  That  effort  achieved  an  MTBR  of  9000  hours  and 
allowed  the  average  Minuteman  I  to  stay  in  the  field  for  over  a  year. 
Though  this  second  development  effort  cost  an  additional  $150  million, 
in  the  long  run  it  saved  some  $1.5  billion,  but  much  more  important,  it 
increased  the  availability  level  of  the  missile  force  from  70  percent  to 
over  95  percent.2 

Carousel  Inertial  Navigation  Subsystem.  In  developing  its 
Carousel  inertial  navigation  subsystem  for  transport  airplanes,  General 
Motors  found  that  it  had  to  use  three  development  efforts.3  After  some 
preliminary  development  work  for  the  Air  Force  with  no  major  concern 
for  MTBR,  General  Motors  signed  a  contract  with  commercial  airlines 
guaranteeing  an  MTBR  of  1500  hours.  It  then  developed  and  produced 
a  model  that  achieved  an  MTBR  of  only  100  hours.  Consequently, 
General  Motors  instituted  a  second  formal  development  effort,  includ¬ 
ing  production  and  operation,  that  yielded  an  MTBR  of  between  500 
and  600  hours.  Still  failing  to  live  up  to  its  contractual  obligation, 
General  Motors  instituted  a  third  development  effort  aimed  exclusively 
at  reliability  improvement.  That  effort  actually  did  achieve  an  MTBR 
of  1500  hours.  The  three  development  efforts  cost  roughly  $50  million 
in  then-current  dollars. 

Although  having  to  operate  under  more  demanding  conditions  than 
either  the  Minuteman  guidance  systems  or  the  Carousel  inertial  navi¬ 
gation  subsystems,  the  F-15’s  inertial  navigation  subsystem  went 


2Cost  estimates  are  expressed  in  then -year  dollars. 

information  about  the  Carousel  program  is  drawn  from  the  personal  files  of  H.  L. 
Shulman,  who  was  responsible  for  the  initial  development  of  Carousel  while  he  was 
Director  of  Research  and  Development  at  the  A.C.  Spark  Plug  Division  of  General 
Motors  Corporation. 
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through  only  one  development  cycle,  yielding  an  MTBR  of  75  hours.4 
The  total  RDT&E  cost  for  the  subsystem  was  $7.6  million  in  then- 
current  dollars.  We  believe  the  F-15’s  medial  navigation  system  wonld 
be  considerably  more  reliable  had  it  been  subjected  to  the  same  kind  of 
rigorous  and  recursive  testing  and  redesign. 

General  Observations 

We  made  certain  general  observations  in  1980,  based  upon  the  fore¬ 
going  experiences  and  twenty  years  of  assisting  the  Air  Force  on  a 
variety  of  ad  hoc  avionics  projects  such  as: 

•  F-4  inertial  navigation  reliability  improvement 

•  F-4  inertial  navigation  support 

•  F-111D  BIT  improvement  program 

•  FB-111  inertial  navigation/Kalman  filter  problem 

•  F-16  avionics  development 

•  F-15  avionics  support 

The  general  observations  are: 

1.  Without  special  data  collection  efforts,  engineers  have  lacked 
data  about  actual  operational  experience,  and  developers  have  been 
hindered  in  identifying  the  most  serious  problems  that  have  plagued 
performance  and  support  of  electronic  subsystems.  Moreover,  too 
often  engineering  information  has  not  focused  on  combat-relevant 
functions  and  has  not  taken  into  account  the  multiple  functions  of  sub¬ 
systems,  the  multiple  operating  states  for  each  function,  and  complex 
integrations  among  subsystems. 

2.  Avionics  acquisition  programs  have  overemphasized  speed  of 
development.  Much  can  and  should  undoubtedly  be  done  to  remove 
many  of  the  bureaucratic  inefficiencies  that  currently  slow  down  the 
acquisition  process.  We  cannot  fight  enemies  with  weapons  that  are 
still  on  the  drawing  boards.  But  an  undue  emphasis  on  speed  of 
development  has  lead  to  a  failure  to 

•  Collect  accurate  and  relevant  data  concerning  potential  problems 
with  sustaining  a  weapon  system’s  designed  performance. 

4The  F-15’s  inertial  navigation  system  must,  for  example,  withstand  repeated  high-g 
environments  and  must  function  properly  in  spite  of  short  warmup  periods.  Although  all 
inertial  guidance  navigation  subsystems  are  extremely  sensitive  to  changes  in  tempera¬ 
ture,  these  changes  are  especially  serious  in  combat  airplanes,  which  must  be  able  to 
scramble  with  short  warning.  By  contrast,  the  inertial  guidance  in  the  Minuteman 
operates  continuously,  and  thus  has  no  need  for  short  warmup;  and  the  inertial  naviga¬ 
tion  subsystems  in  transports  can  usually  be  warmed  up  adequately  before  the  transport 
needs  to  take  off. 
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•  Redesign  portions  of  the  weapon  system  and  its  support  system 
to  avoid  these  problems 

When  fielding  new  weapon  systems,  we  have  been  preoccupied  with  the 
time  to  IOC  and  have  largely  ignored  time  to  matured  operational 
capability.  The  latter  can  take  much  longer,  as  can  be  seen  in  the  time 
needed  to  introduce  radar  R&M  improvements  for  the  F-15  and  F-16 
(see  Sec.  IV). 

CONCEPT 

Drawing  upon  the  very  positive  experience  of  the  Minuteman  I  guid¬ 
ance  and  the  Carousel  navigation  programs  and  a  series  of  ad  hoc 
avionics  projects  over  the  years,  we  proposed  the  following  concept  for 
maturational  development  during  1980. 

Shortly  after  introducing  the  first  24  to  72  units  of  a  new  subsystem 
to  an  operational  environment,  a  new  phase  of  full-scale  engineering 
development  would  commence.  This  new  phase,  known  as  the  matura¬ 
tional  development  phase,  would  consist  of  two  stages: 

•  Assessment  of  R&M  situation. 

•  Implementation  of  improvements. 

Stage  1:  Assessment  of  R&M  Situation 

The  objective  of  stage  1  is  for  equipment  contractors  to  work  with 
the  government  in  a  team  effort  to  define  a  comprehensive  package  of 
R&M  improvements  that  addresses  the  most  serious  R&M  deficiencies 
in: 

•  The  subsystem  of  interest. 

•  The  interfaces  among  the  subsystem  of  interest,  the  host 
weapon  system,  and  related  subsystems. 

•  The  support  process  at  all  levels  (cockpit,  flightline,  shop,  and 
depot)  for  the  subsystem  of  interest.  This  includes  tests,  test 
equipment,  and  maintenance  instructions  (TOs). 

Accomplishment  of  the  objective  requires  four  major  activities,  each 
requiring  considerable  involvement  by  equipment  contractors: 

1.  Collect  data  based  on  operational  experience  to  determine 
where,  how,  how  often,  and  why  a  combat-essential  avionics 
subsystem  fails  to  deliver  the  full  amount  of  its  designed  capa¬ 
bility. 
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2.  Analyze  the  data  to  identify  the  most  serious  deficiencies  in 
terms  of  the  effect  on  dependable  delivery  of  the  subsystem’s 
designed  capability. 

3.  Define  candidate  actions  to  correct  or  remediate  the  most  seri¬ 
ous  deficiencies  and  analyze  the  prospective  cost  and  benefits. 

4.  Work  with  the  government  to  define  a  comprehensive  package 
of  actions  to  improve  the  R&M  situation  for  the  subsystem  of 
interest. 

Data  Collection.  Three  critical  considerations  are  the  nature  of 
the  data  to  be  collected,  who  collects  the  data,  and  the  size  of  the  data¬ 
base. 

Data  must  be  sufficient  to  allow  personnel  to  assess  the  effectiveness 
of  each  maintenance  action  based  upon  tracking  operational  perfor¬ 
mance  after  the  completion  of  maintenance.  To  do  this: 

•  Data  on  all  indications  of  potential  problems  must  be  collected 
at  the  pilot’s  maintenance  debriefing. 

•  Data  on  all  maintenance  actions  must  be  collected  at  the  flight¬ 
line,  the  shop,  and  the  depot. 

Only  the  equipment  contractors  have  the  engineering  expertise  and 
knowledge  of  the  equipment  essential  to  the  design  of  R&M  improve¬ 
ments.  To  assure  that  appropriate  data  are  acquired  to  support  their 
analysis  and  subsequent  redesign  efforts  the  equipment  contractors 
must  field  and  manage  the  data  collection  team. 

The  database  must  be  large  enough  to  capture  a  statistically  mean¬ 
ingful  sample  for  the  most  serious  R&M  deficiencies.  Adequate  data 
samples  are  essential  to  understanding  which  deficiencies  are  most 
prevalent.  One  way  to  gauge  the  necessary  size  for  the  database  for  a 
subsystem  of  interest  is  to  consider  the  average  number  of  flights 
between  visits  to  the  depot  for  the  types  of  equipment  (SRU  or  LRU) 
that  most  frequently  go  to  the  depot.  For  such  equipment,  it  is  desir¬ 
able  to  have  at  least  20  visits  in  order  to  sort  out  the  dominant  failure 
modes.  For  example,  say  that  a  subsystem  has  SRUs  that,  on  average, 
visit  the  depot  no  more  than  once  every  300  flights  for  any  given  type. 
This  number  is  representative  for  complex  subsystems.  Then,  with  our 
assumptions,  the  database  should  be  based  on  at  least  6000  flights  just 
to  have  a  good  sample  for  the  SRUs  that  failed  most  frequently.  To 
gain  statistically  meaningful  insight  into  other  SRUs  it  is  probably 
prudent  to  double  or  triple  this  figure.  Thus,  for  modern  avionics,  our 
sense  is  that  a  database  should  cover  from  12,000  to  18,000  flights. 

Analysis.  A  critical  consideration  here  is  the  measures  used  to 
characterize  R&M  deficiencies.  Currently,  mean  time  between  failures 
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(MTBF)  is  the  most  widely  accepted  measure  of  reliability.  For  avion¬ 
ics,  MTBF  translates  into  mean  flying  hours  between  failures  con¬ 
firmed  by  the  avionics  shop.  MTBF  data  unfortunately  fail  to  capture 
the  entire  R&M  picture  because  they  do  not  reflect  occasions  when: 

•  The  pilot  does  not  report  discrepancies. 

•  The  pilot  detects  something  wrong  with  avionics  equipment,  but 
maintenance  personnel  do  not  remove  an  LRU  from  the  air¬ 
plane. 

•  The  BIT  indicates  something  wrong  with  avionics  equipment, 
but  maintenance  personnel  do  not  remove  an  LRU  from  the 
airplane. 

•  Maintenance  personnel  remove  an  LRU  from  the  airplane  but 
cannot  find  anything  wrong  with  it  in  the  shop. 

To  arrive  at  a  much  more  complete  view  of  the  R&M  picture,  we 
need  to  recognize  that  R&M  is  a  quality  with  multiple  dimensions. 
Moreover,  because  precision  becomes  increasingly  more  difficult  as  the 
complexity  of  the  equipment  grows,  we  can  only  devise  indicators  that 
capture  the  salient  features  of  the  R&M  quality  being  sought.  In  view 
of  these  limitations,  we  suggest  a  more  complete  assessment  of  R&M 
that  includes  both  MTBF  and  fault  removal  efficiency.  To  estimate  the 
fault  removal  efficiency,  we  need  both  MTBF  and  the  mean  time 
between  indications  (MTBI)  of  faults  perceived  by  the  pilot  and  by  the 
BIT. 

To  arrive  at  a  meaningful  MTBI  for  current  aircraft,  pilots  need  to 
report  all  indications  of  suspected  faults,  maintenance  personnel  need 
to  maintain  a  historical  record  of  all  pilot-reported  faults  for  each  air¬ 
craft,  and  the  Air  Force  needs  to  select  aircraft  randomly  for  those  rare 
opportunities  when  air-to-air  weapons  are  fired  and  ECM  equipment  is 
employed.  Moreover,  to  arrive  at  a  meaningful  MTBI  for  future  air¬ 
craft,  the  BIT  should  capture  and  report  information  about  operating 
modes  (such  as  switch  settings)  and  operating  environments  (such  as 
temperature  of  equipment)  whenever  the  BIT  detects  a  fault. 

If  all  indications  are  reported,  differences  between  MTBI  and 
MTBF  reflect  difficulties  in  locating  and  correcting  faults  in  avionics 
equipment.  We  propose  the  following  avionics  maintainability  indica¬ 
tor: 


Fault 

Removal 

Efficiency 


MTBI 

-  x  100 

MTBF 
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where 

MTBI 

-  Mean  Time  Between  flights  with  one  or  more 
Indications  of  faulty  operation  of  the  avionics 
subsystem. 

MTBF 

=  Mean  Time  Between  flights  with  a  shop-con¬ 
firmed  Failure  of  the  avionics  subsystem. 

A  low  indicator  means  that  maintenance  personnel  can  rarely  find  a 
confirmed  failure  for  flights  when  one  or  more  faults  were  indicated. 
This  points  toward  problems  in  one  or  more  of  the  avionics  subsystem, 
the  BIT,  the  shop/depot  tests/equipment,  the  TOs,  and  maintenance 
personnel  training. 

Such  an  avionics  maintainability  indicator  will  identify  those  sub¬ 
systems  that  experience  the  most  difficulty  in  meeting  their  designed 
capability  and  that  impose  the  greatest  demands  on  the  support  pro¬ 
cess.  Subsystems  that  experience  especially  great  problems  are  lacking 
in  maturity:  They  and  their  associated  support  process  have  not 
experienced  enough  engineering  development  to  regularly  deliver  their 
full  measure  of  designed  capability  within  the  current  support  struc¬ 
ture.  The  data  collection  and  analysis  activities,  including  the  applica¬ 
tion  of  this  maintainability  indicator,  point  to  the  areas  in  most  need 
of  improvement.5 

Candidate  Improvements.  To  identify  the  most  appropriate  ways 
to  achieve  needed  improvements,  the  equipment  contractors  may  need 
to  review  whether  certain  materials  should  be  changed,  certain  com¬ 
ponents  should  be  derated,  and  other  components  should  be  redesigned. 
In  addition,  they  may  have  to  consider  imposing  tighter  quality  control 
on  certain  materials  and  production  methods.  To  improve  maintain¬ 
ability,  the  contractors  may  have  to  change  the  BIT  circuitry  and  pro¬ 
grams,  the  system  partitionings,  and  the  test  points.  In  addition,  they 
may  have  to  add  test  points  and  capture  more  information  about  the 
airplane’s  mode  of  use  at  the  time  of  degraded  performance.  All  of  this 
should  occur  not  only  at  the  subsystem  and  LRU  level  (involving  the 
subsystem  contractor)  but  also  at  the  weapon-system  level  (involving 
the  prime  contractor)  when  the  equipment  is  installed  and  tested  in  a 
flight  vehicle. 


5If  there  are  many  faUe  indications  (e.g.,  because  of  a  bad  BIT)  then  the  fault 
removal  efficiency  will  have  a  low  value.  By  improving  the  BIT  so  as  to  reduce  the 
number  of  false  indications,  the  contractor  can  raise  the  fault  removal  efficiency  for  his 
subsystem.  This  parameter  intentionally  penalizes  a  bad  BIT  because  it  undermines 
maintainability. 
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Improvement  Package.  The  last  activity  for  Stage  1  is  the  formu¬ 
lation  and  coordination  of  a  proposed  R&M  improvement  package 
through  these  organizations.  Formulation  of  a  cohesive  and 
comprehensive  package  of  improvements  will  often  require  the  coopera¬ 
tion  of  the  subsystem  contractor,  the  shop  test  equipment  contractor, 
the  depot  t«.*st  equipment  contractor(s),  and  the  host  weapon  system 
contractor.  It  will  also  require  the  cooperative  efforts  of  the  System 
Program  Office  for  the  host  weapon  system,  the  Air  Logistics  Center 
for  the  subsystem  of  interest,  and  the  Air  Logistics  Center  for  the  test 
equipment. 


Stage  2:  Implementation  of  Improvements 

Stage  2  of  maturational  development  involves  carrying  out  the 
improvements,  some  of  which  can  be  expected  to  require  further 
development  efforts. 

Without  such  further  development,  engineers  cannot  accurately 
assess  where,  how,  how  often,  or  why  many  Type  A  and  Type  B 
failures  will  occur  in  avionics  subsystems.  This  is  true  for  several  rea¬ 
sons:  Their  design  methods  and  tools  simply  cannot  predict  all  impor¬ 
tant  causes  and  modes  of  failures;  they  lack  complete  information 
about  new  materials  and  processes  needed  to  manufacture  avionics  sub¬ 
systems;  and  they  cannot  foresee  all  environments  in  which  the  avion¬ 
ics  subsystems  will  perform  and  all  complex  interrelations  among  them. 

Developers  therefore  need  to  conduct  a  development  effort,  including 
data  collection  and  operational  tests,  at  least  once  to  discover  where, 
how,  how  often,  and  why  a  subsystem  will  fail  (see  Fig.  3.1).  After 
identifying  these  failures,  developers  must  then  repeat  a  portion  of  the 
engineering  development  effort  if  they  are  to  reduce  failure  rates  and 
improve  maintainability. 


IMPLEMENTATION  OF  MATURATIONAL 
DEVELOPMENT 

The  experiences  of  the  Air  Force  with  its  Minuteman  missile  guid¬ 
ance  subsystem  and  of  General  Motors  with  its  Carousel  inertial  navi¬ 
gation  subsystem  for  transports  suggest  that  a  maturational  develop¬ 
ment  process  can  achieve  dramatic  improvements.  Inertial  guidance  or 
navigation  subsystems  on  missiles,  transports,  and  tactical  fighters 
share  similar  basic  components:  accelerometers,  gyros,  and  electronics. 
In  addition,  they  share  similar  problems  with  their  reliability  and 
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maintainability.  However,  tactical  fighters  must  operate  in  much  more 
demanding  environments.  We  recommend  a  formal  maturational 
development  phase  for  three  classes  of  complex  combat-essential  avion¬ 
ics  subsystems: 

•  New  subsystems  that  are  just  beginning  development. 

•  Existing  subsystems  that  are  being  modified  to  improve  their 
functional  performance. 

•  Existing  subsystems  in  which  improvements  in  reliability  and 
maintainability  would  considerably  narrow  the  gap  between 
designed  and  operationally  available  performance. 

Maturational  development  can  achieve  the  largest  benefit-to-cost 
ratio  when  aimed  at  developing  selected  new  avionics  subsystems  that 
are  just  beginning  development.  In  such  cases,  it  should  occur  prior  to 
high-rate  production  to  avoid  the  high  costs  of  retrofitting  hardware. 
For  this  class  of  equipment,  the  Air  Force  might  most  profitably  begin 
with  new  avionics  for  the  next  generation  of  tactical  fighters. 

In  addition,  a  maturational  development  phase  can  greatly  improve 
selected  existing  avionics  equipment  whenever  it  receives  large 
performance-oriented  improvements.  Existing  equipment  can  most 
benefit  from  maturational  development  when  its  improvements: 

•  Involve  large  changes. 

•  Affect  the  weapon  system’s  overall  design. 

•  Increase  the  airplane’s  modes  of  operation. 

•  Raise  performance  specifications  considerably  for  existing 
modes. 

•  Make  a  large  proportion  of  its  components  no  longer  inter¬ 
changeable  with  those  of  the  previous  model. 

With  this  class  of  equipment,  the  Air  Force  should  especially  consider 
instituting  maturational  development  for 

•  Fire  control  radars,  such  as 

—  APG  68  for  the  F-16  C/D. 

—  APG  70  for  the  F-15  E. 

•  Electronic  counter  measures  equipment  such  as 
—  Internal  countermeasures  set  for  the  F-15. 

—  Threat  warning  system  for  the  F-16. 

—  Advanced  self-protection  jamming  system  for  the  F-16. 

—  Jamming  pods  (ALQ-131  Block  2  and  ALQ-184)  for  the  F-4 
and  the  F-lll. 

•  Improved  weapon  delivery  systems  for  the  F-15  and  F-16. 

•  New  low  altitude  night  targeting  infrared  (LANTIRN)  navigation 
and  targeting  pods. 
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Finally,  a  maturational  development  effort  can  also  help  address 
reliability  and  maintainability  problems  experienced  by  equipment 
already  in  the  field.  For  this  class  of  equipment,  the  Air  Force  has 
already  begun  addressing  currently  fielded  models  of  the  fire  control 
radars  in  both  the  F-15  C/D  and  the  F-16  A/B  (see  Sec.  IV). 

Maturational  development  can  lead  to  both  mid-term  and  long-term 
improvements.  For  the  mid-term,  it  can  help  reduce  the  adverse 
effects  of  problems  with  R&M  on  combat  effectiveness.  For  the  long 
term,  it  can  help  avoid  these  problems  before  the  Air  Force  adds  new 
avionics  equipment  to  future  tactical  airplanes. 


CONCERNS  AND  SEARCHES  FOR  ALTERNATIVES 

Since  the  concept  of  a  formal  maturational  development  phase  was 
proposed  in  1980,  concerns  about  its  practicality  have  stimulated  a 
search  for  alternatives.  Four  classes  of  concerns  have  emerged: 

•  Cost. 

•  Schedule. 

•  Program  survival. 

•  Management  of  subsystem  development. 

Cost 

The  concept  of  maturational  development  does  call  for  additional 
investment  by  the  government  during  both  Stage  1  and  Stage  2.  Such 
additional  investment  needs  to  be  weighed  against  the  prospective 
benefits  delivered  by  more  mature  subsystems  and  a  more  mature  sup¬ 
port  process.  Because  of  the  cost,  maturational  development  should  be 
selectively  applied — both  in  terms  of  the  subsystems  selected  and  the 
timing  of  subsystem  development.  Because  the  costs  of  retrofitting 
hardware  can  be  the  dominant  expense  in  a  maturational  development 
program  (as  demonstrated  in  Sec.  IV),  maturational  development 
promises  to  be  most  beneficial  when  Stage  1  (Assessment)  can  be  com¬ 
pleted  very  easily  during  the  production  run.  Starting  Stage  1  shortly 
after  IOC  would  help  reduce  costs,  as  would  stretching  out  the  produc¬ 
tion  run.  Another  cost -saving  tactic  would  be  to  develop  subsystems 
for  application  to  multiple  weapon  systems  where  practical.  The 
maturation  cost  would  thereby  be  amortized  across  a  broader  produc¬ 
tion  base. 

Stretching  out  production  raises  other  concerns,  such  as  increased 
program  cost  due  to  the  stretch  and  delaying  the  introduction  of  new 
capabilities.  The  benefits  of  stretching  production  will  depend  upon 
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the  nature  of  the  subsystem  development  program  and  the  develop¬ 
ment  program  for  the  host  weapon  system. 

Schedule 

The  maturational  development  concept  does  not  have  to  influence 
development  schedules  for  either  subsystems  or  host  weapon  systems, 
unless  the  government  makes  the  judgment  that  it  would  be  wise  to 
start  subsystem  development  early  or  stretch  production.  Presumably, 
a  decision  to  do  so  would  be  based  upon  a  balanced  consideration  of 
both  the  prospective  benefits  and  the  costs6  of  such  a  choice.  Here  the 
central  question  would  be  whether  it  is  worth  starting  a  little  sooner 
(or  waiting  a  little  longer)  to  obtain  a  more  mature  subsystem  and  a 
more  mature  support  process. 

Of  course,  starting  development  sooner  has  another  hidden  oppor¬ 
tunity  cost  in  that  an  earlier  start  on  design  implies  an  earlier  freeze 
on  the  technology  base  to  be  incorporated  in  the  design.  Here  the 
choice  is  between  a  somewhat  newer  technology  and  a  less  mature  sub¬ 
system  and  a  somewhat  older  technology  and  a  more  mature  subsys¬ 
tem.  Such  difficult  choices  would  need  to  be  carefully  considered  for 
each  major  subsystem  that  would  be  a  candidate  for  maturational 
development. 

Program  Survival 

Regardless  of  whether  maturational  development  is  scheduled  in  a 
way  that  influences  a  subsystem  (or  weapon  system)  acquisition 
schedule,  there  is  a  concern  that  planning  the  funds  and  designating 
milestones  for  a  maturational  development  phase  will  raise  eyebrows 
and  jeopardize  the  survival  of  a  program.  For  maturational  develop¬ 
ment  to  be  accepted  and  practiced  effectively,  a  new  mindset  will  be 
required  throughout  all  levels  of  the  acquisition  process.  The  explicitly 
planned  incorporation  of  a  maturational  development  phase,  with 
healthy  funding  plans  for  both  Stage  1  and  Stage  2,  should  be  viewed 
as  a  sign  of  a  healthy  program.  Development  programs  that  cannot 
afford  to  conduct  a  healthy  maturational  development  phase  should  be 
carefully  reconsidered. 


6Costs,  of  course,  would  include  both  fiscal  expenses  and  the  opportunity  cost  of 
delayed  fielding  of  equipment. 
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Subsystem  Development 

For  the  past  three  decades  the  Air  Force  has  relied  heavily  on  each 
weapon  system’s  prime  contractor  to  oversee  and  manage  the  develop¬ 
ment  of  the  complex  and  critical  subsystems  used  in  its  weapon  sys¬ 
tem.  Because  the  Air  Force  has  not  had  much  recent  experience  in 
managing  the  development  of  complex  avionics  subsystems,  there  is  a 
concern  that  it  would  have  neither  the  resources  nor  the  experience  to 
adequately  perform  this  critical  function. 

A  further  concern  is  that  government’s  management  of  subsystem 
development  would  limit  the  ability  of  a  prime  contractor  to  optimize 
the  overall  performance  of  the  weapon  system.  If,  for  example,  a  fire 
control  radar  was  government-furnished  equipment,  then  the  prime 
contractor  would  be  stuck  with  the  general  arrangement  of  the  subsys¬ 
tem  and  would  not  be  free  to  explore  tradeoffs  in  terms  of  the  packag¬ 
ing  and  placement  of  the  equipment. 

The  concept  of  maturational  development  does  not  necessarily 
require  the  government  to  manage  subsystem  development,  which 
would  clearly  have  certain  costs.  Such  costs  and  the  prospective  bene¬ 
fits  would  need  to  be  carefully  weighed. 

Searches  for  Alternatives 

The  foregoing  concerns  have  prompted  searches  for  a  more  satisfac¬ 
tory  alternative  to  maturational  development.  Attention  has  have 
directed  toward  three  areas: 

•  Existing  maintenance  data. 

•  Warranties. 

•  Requirements  and  testing. 

Recent  RAND  research  has  identified  numerous  ways  that  the  Air 
Force  can  work  toward  strengthening  its  activities  in  each  of  these 
areas.7  However,  even  when  strengthened,  such  activities  only  partly 
address  the  weaknesses  that  maturational  development  strives  to 
remedy. 

Existing  Maintenance  Data.  Although  the  Air  Force  can  learn 
more  from  the  maintenance  data  that  it  routinely  collects,8  the  limited 
depth  of  that  data  renders  it  insufficient  for  resolving  root  causes  of 
R&M  deficiencies  for  complex  avionics.  It  can  help  identify  problem 
areas,  but  it  fails  to  provide  the  detail  necessary  for  engineering 

7Petruschell,  Smith,  and  Kirkwood,  1987;  Stanley  and  Birkler,  1986;  Stucker  and 
Smith,  1987. 

8See,  for  example,  Stucker  and  Smith,  1987. 
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analysis,  especially  for  Type  B  faults.  Such  analysis  is  essential  to 
assessing  the  full  scope  and  effect  of  R&M  problems,  as  well  as  to 
understanding  root  causes.  Although  the  symptoms  of  R&M  problems 
discussed  in  Sec.  IV  were  also  evident  in  the  Air  Force’s  maintenance 
data,  there  was  great  ambiguity  about  underlying  causes9  until  the 
engineering  databases  and  engineering  analyses  were  completed. 

Warranties.  Application  of  warranties  to  the  acquisition  of  weapon 
systems  became  mandatory  during  1984-1985.  The  most  beneficial  role 
and  probable  contributions  of  such  warranties  is  a  subject  in  need  of 
further  study.  The  general  types  of  information  that  the  Air  Force 
needs  to  collect  to  support  such  research  has  been  identified.10 

Analysis  of  the  limited  information  that  is  available  for  pre-1985 
warranties  “suggested  that  at  least  four  factors  contribute  positively  to 
warranty  success: 

•  Specific,  easily  measurable  objectives; 

•  Simple,  explicit  contractor  incentives  and  remedies; 

•  Simple,  explicit  government  duties;  and 

•  Reasonable  prices  and  expectations.”11 

It  is  difficult  to  extrapolate  the  ultimately  most  desirable  role  of  war¬ 
ranties  from  this  initial  assessment.  The  apparent  message,  however, 
is  to  keep  things  simple  and  explicit.  This  indication  raises  some  early 
warning  signs  for  complex  avionics  equipment.  The  complexities  of 
both  the  airborne  equipment  and  its  associated  support  process  create 
reasonable  doubts  about  whether  the  Air  Force  can  devise  an  adequate 
set  of  simple  measures  that  would  provide  explicit  contractor  incen¬ 
tives,  while  delineating  explicit  government  duties. 

Unfortunately,  the  fault-removal-efficiency  measure,  introduced  pre¬ 
viously  in  this  section,  does  not  appear  to  satisfy  these  guidelines.  Not 
only  is  the  value  for  this  measure  influenced  by  the  performance  of 
various  contractors  and  various  government  agencies,  but  sorting  out 
the  probable  contributions  of  each  contractor  or  agency  can  require  a 
cooperative  engineering  effort.12  It  seems  to  us  that  future  research  on 
warranties  should  devise  sets  of  measures  and  approaches  to  warranties 
that  are  conducive  to  such  cooperation. 

Consequently,  for  both  radars  there  was  much  disagreement  about  responsibility  for 
the  root  causes.  For  example,  frequent  removal  of  one  type  of  LRU  was  claimed  to  be 
due  to  poor  troubleshooting  and  problems  in  other  parts  of  the  airplane.  Only  the 
contractor’s  engineering  analysis  finally  adjudicated  this  issue.  (See  Sec.  IV.) 

l0See  Stucker  and  Smith,  1987. 

"Ibid.,  p.  vi. 

l2See,  for  example,  the  efforts  described  in  Sec.  IV. 
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Requirements  and  Testing.  Simple  measures,  with  explicit  and 
contractor-specific  incentives,  are  also  essential  to  strengthening 
requirements  and  testing.  RAND  research  has  identified  a  broad  range 
of  opportunities  for  improving  requirements  and  testing  to  help 
increase  the  operational  suitability  of  future  weapon  systems.13  A  major 
recommendation  is  to 

Expand  contractual  accountability  for  suitability-related  characteris¬ 
tics.  This  requires  (1)  using  broader  R&M  specification  measures 
that  more  fully  reflect  system  and  subsystem  maintenance  demands 
that  drive  the  Air  Force  support  burden,  (2)  using  compliance-test 
ground  rules  that  do  not  compromise  those  contractual  specifications 
through  excessive  exclusions  of  particular  kinds  of  failure  events,  (3) 
conducting  compliance  tests  of  sufficient  duration  and  scope  to  have 
confidence  the  performance  measured  is  representative  of  the  system 
at  the  time  of  the  test,  (4)  performing  more  system-level  compliance 
testing  on  production  equipment,  (5)  conducting  compliance  tests  of 
system  characteristics  that  contribute  to  resilience  to  attack  and 
mobility,  and  (6)  including  contractual  mechanisms  using  obligations 
and  incentives  to  motivate  contractors  to  meet  more  stringent  suit¬ 
ability  requirements.14 

As  with  warranties,  the  beneficial  contribution  of  expanded  contrac¬ 
tual  accountability  for  R&M  will  be  influenced  largely  by  the  measures 
employed  in  contracts.  For  complex  avionics  little  progress  has  been 
made  in  the  area  of  developing  measures  that 

•  Provide  appropriately  comprehensive  coverage  of  reliability  and 
maintainability. 

•  Lend  themselves  to  simple  and  explicit  delineation  of  responsi¬ 
bilities. 

As  already  noted,  the  fault-removal-efficiency  measure  fails  to  satisfy 
the  later  requirement. 

Given  continuing  limitations  in  both  the  existing  maintenance  data 
and  the  measures  that  we  have  for  characterizing  R&M,  we  fail  to  find 
an  attractive  alternative  to  maturational  development.  We  believe  the 
key  to  such  a  discovery  would  lie  in  the  invention  of  measures  that  can 
simultaneously  provide  the  detail  required  by  engineers  and  the  sim¬ 
plicity  and  explicitness  needed  for  effective  contracting. 

Of  course,  maturational  development  is  not  without  its  burdens. 
The  two  bottom  line  questions  are: 


13Stanley  and  Birkler,  1986. 
14Ibid.,  p.  100. 
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•  Are  the  prospective  benefits  worth  the  costs? 

•  If  so,  what  are  the  most  effective  ways  to  apply  the  concept? 

To  help  answer  these  questions,  two  exploratory  applications  of  the 
concept  were  approved  during  June  1981.  The  results  of  Stage  1  of 
these  programs,  launched  during  June  1984,  are  described  in  the  next 
section. 


IV.  EXPLORATORY  APPLICATIONS  OF 
MATURATION AL  DEVELOPMENT  TO 
THE  F-16  AND  F-15  RADARS 


Two  exploratory  applications  of  the  maturational  development  con¬ 
cept  were  launched  during  June  1984  with  the  objectives  of  learning 
lessons  about  the 

1.  Potential  value  of  the  concept. 

2.  Effective  procedures  for  applying  the  concept. 

Further  objectives  for  the  effort  were  to: 

3.  Better  assess  the  R&M  state  of  health  for  the  subject  subsys¬ 
tems. 

4.  Define  the  most  beneficial  opportunities  for  improving  the  R&M 
situation  for  these  subsystems. 

This  section  describes  the  technical  results  for  the  last  two  objectives 
and  then  draws  lessons  from  those  experiences  to  meet  the  first  two 
objectives. 


THE  EXPLORATORY  APPLICATIONS 

Stage  1  of  the  exploratory  application  has  been  completed  for  the 
APG-66  radar  used  by  the  F-16  A/B  and  the  APG-63  radar  used  by  the 
F-15  C/D.  The  technical  results  fall  into  three  categories: 

•  Results  specific  to  the  F-16  radar. 

•  Results  specific  to  the  F-15  radar. 

•  Results  common  to  both  radars. 

The  Stage  1  results  are  the  product  of  a  joint  effort  by  the  Air  Force, 
RAND’s  Project  AIR  FORCE,  and  industry.  The  Air  Force  designated 
the  Strike  Systems  Program  Office  (SPO),  within  the  Aeronautical 
Systems  Division,  to  manage  the  Stage  1  effort  for  both  radars.  RAND 
worked  closely  with  the  Strike  SPO  in  formulating  and  guiding  the 
Stage  1  effort,  which  became  known  as  the  F-15/F-16  Radar  R&M 
Improvement  Program. 
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RESULTS  SPECIFIC  TO  THE  F-16  RADAR 

Stage  1:  Assessment  of  R&M  Situation 

The  Strike  SPO  contracted  with  the  Westinghouse  Defense  and 
Electronic  Systems  Center  for  the  four  major  activities  constituting 
Stage  1  of  maturational  development: 

•  Data  Collection. 

•  Analysis. 

•  Candidate  Improvements. 

•  Formulation  of  an  Improvement  Package. 

The  one-year  contract  for  approximately  $6  million  contained  provi¬ 
sions  under  which  Westinghouse  contracted  with  General  Dynamics 
(the  F-16  weapon  system  prime  contractor)  for  engineering  assistance 
in  carrying  out  these  activities. 

Data  Collection.  Data  on  the  F-16’s  radar  were  collected  from  150 
F-16s  monitored  during  a  six-month  period  (June-December  1984)  at 
Hill  AFB  and  Hahn  Air  Base.  These  data  covered  16,077  flights  and 
the  resulting  radar  maintenance  at  these  bases  and  the  depot.  Con¬ 
tractor  personnel  interviewed  pilots  after  maintenance  debriefing,  docu¬ 
menting  all  flight  anomalies  observed  by  the  pilot,  including  those  indi¬ 
cated  by  the  BIT.  Contractor  personnel  also  documented  maintenance 
on  the  flightline,  in  the  shop,  and  at  the  depot.  This  effort  required 
the  fielding  of  a  data  collection  team  that  included  32  people  provided 
by  the  radar  contractor  (Westinghouse)  and  four  people  provided  by 
the  weapon  system  prime  contractor  (General  Dynamics). 

Analysis.  As  Fig.  4.1  shows,  the  radar  for  the  F-16  A/B  has  six 
LRUs.  It  has  tremendous  versatility  and  can  provide  many  functions 
in  air-to-air  and  air-to-ground  combat.  Pilots  can  set  switches  on  the 
control  panel  (see  upper  right),  the  throttle  grip,  and  the  side  stick 
controller  in  over  12,000  unique  configurations.  Although  many  pilots 
use  the  same  basic  modes,  the  versatility  of  the  radar  enables  them  to 
develop  individual  sets  of  switch  settings  that  govern  antenna  scan  pat¬ 
terns,  transmitter  options,  and  display  options. 

This  functional  versatility  means  that  pilots  using  the  same  radar 
have  the  opportunity  to  use  it  in  very  different  ways  from  one  flight  to 
another.  As  a  result,  a  pilot  on  one  flight  may  see  a  fault  indicated 
that  a  pilot  on  the  next  flight  does  not  see. 

For  the  F-16,  the  main  R&M  problems  arise  with  the  Low  Power 
Radio  Frequency  (LPRF)  LRU  in  the  upper  left  of  Fig.  4.I.1  This  unit 
generates  microwave  signals  that  the  transmitter  radiates  at  various 


'The  LPRF  combines  functions  of  several  LRUs  on  the  F-15  radar. 
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radio  frequencies.  It  also  processes  the  returning  microwave  signals 
captured  by  the  antenna.  This  processing  is  analog  rather  than  digital 
and  involves  delicate  radio  frequency  circuitry. 

The  extent  and  severity  of  the  R&M  problems  with  the  LPRF  were 
determined  and  validated  by  a  four-step  process  that  included  special 
data  collection,  engineering  tests,  engineering  analysis,  and  indepen¬ 
dent  review. 

Figure  4.2  helps  show  why  a  special  data  collection  effort  was 
needed.  Information  vital  to  engineering  analysis  would  otherwise  be 
unavailable.  For  example,  only  information  such  as  that  in  the  shaded 
rows  was  being  entered  into  the  standard  Air  Force  data  systems. 
Information  in  the  remaining  rows  was  unavailable  even  to  radar 
maintenance  technicians.  It  was  only  available  to  the  radar  contractor 
and  only  because  there  was  a  special  contractor  team  gathering  such 
information. 


41 


Date 

BIT 

Pilot  Comments 

Radar 

LRU 

Removed 

Op. 

Code 

June  20 

Med  PRF  false  targets  in  Air  Mode 

1 

25 

032 

035 

MFL  then  screen  flooded  with  false 
targets 

—Recycled  &  worked  OK  MFL  at 
touchdown 

OK 

1 

26 

057  :S 
035 
032 
048 

mmMmmmmmmMmi 

S  Inop  > 

•if  2 

(CND) 

27 

' 

Very  few  retums/would  not  lock  on 
to  any  tgts  use  mostly  A/A.  No  returns 
in  AGR. 

OK 

1 

27 

035 

032 

340 

048 

RDR  mop  —  no  returns  Failed 

5  min.  after  T.O.  only  used  A/A 
mode 

Sj  Inop  s-i 

3 

XMTR 
:  0763 

S:S>:  Ss-sSs  i:; 

28 

False  targets 

Degrd 

S'  2  : 

(CND) 

July  10 

ECM  pods  used,  —lot  of  false  tgts  & 
chevrons.  ACM  &  slewabte  trouble 
locking 

Degrd 

1 

10 

False  tgts  &  chevrons.  Radar  good 

OK 

1 

11 

055 

OK 

1 

.  13 

False  targets/would  not  hold  lock 

O  band  noise  bar  missing 

Degrd 

2 

(CND) 

13 

Numerous  false  targets 

24 

Radar  slow  to  lock  on  —  had  some 
false  targets  &  chevrons 

Degrd 

1 

29 

Rdr  w/n  lock  until  17-18  miles-m/ 
then  break,  relock  &  worked  OK 

OK 

1 

30 

Had  lots  of  false  targets 

Degrd 

1 

31 

Numerous  false  tgts  throughout  flight 

Degrd 

1 

■:  Aug  2 

Radar  cursors  were  stuck  on 

ok  : 

S  3 

(CND) 

2 

Numerous  false  targets 

Degrd 

1 

6 

001 

OK 

1 

Fig.  4.2— Performance  of  radar  in  F-16  number  0577 
(June  20  through  August  6,  1984) 
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This  figure  illustrates  three  maior  points: 

1.  Pilots  do  not  request  maintenance  for  every  indication  of  a  fault 
in  the  radar.  A  request  for  maintenance  is  influenced  by  many 
considerations  including: 

—  What  does  the  airplane  need  to  do  on  its  next  training 
flight? 

—  Would  the  indicated  fault  affect  the  training  value  of  the 
next  flight? 

—  What  are  the  chances  that  the  indicated  fault  was  caused  by 
microwave  signals  generated  by  something  external  to  the 
aircraft? 

—  What  is  the  likelihood  that  technicians  can  duplicate  the 
fault  on  the  ground? 

—  If  the  fault  is  duplicated,  what  are  the  chances  that  a 
replacement  spare  is  available? 

2.  Standard  data  systems  miss  many  indications  of  degraded  radar 
performance.  This  is  because  information  about  performance 
degradations  is  regularly  collected  only  when  a  pilot  requests 
maintenance  (flights  that  returning  pilots  label  Code  2  or  Code 
3).  However,  a  history  of  degraded  performance  may  be  develop¬ 
ing  even  when  pilots  are  not  requesting  maintenance  (Code  1). 

3.  The  F-16’s  BIT  often  fails  to  detect  faults,  especially  Type  B 
faults.2  Numbers  in  the  second  column  refer  to  specific  BITs 
that  indicate  failure  of  the  radar.  Some  faults,  such  as  those 
causing  false  targets  and  lock -on  problems,  may  not  be  caught 
by  BITs.  Other  problems,  such  as  a  faulty  transmitter,  are  only 
sometimes  caught  by  BITs.  In  Fig.  4.2,  for  example,  the  first 
BIT  indication  of  a  fault  in  the  transmitter  occurred  on  June  25, 
when  tests  032  and  035  failed.  Although  the  pilot  also  observed 
indications  of  a  problem  with  false  targets,  he  did  not  request 
maintenance.  BIT  indications  of  a  fault  in  the  transmitter 
occurred  again  on  the  next  day,  and  that  pilot  requested  mainte¬ 
nance.  However,  the  radar  specialist  Could  Not  Duplicate 
(CND)  the  BIT  failures  reported  by  the  pilot,  and  the  technician 
did  not  remove  the  radar  transmitter.  The  first  flight  the  fol¬ 
lowing  day  yielded  no  reports  of  any  BIT  failures.  But  the  next 
flight  did,  and  the  BIT  failures  again  pointed  to  a  faulty 
transmitter.  Moreover,  the  pilot  assigned  a  Code  3  to  the 
maintenance  request.  Maintenance  then  removed  the 
transmitter  (XMTR)  and  sent  it  to  the  shop  for  different  (and 
more  detailed)  testing  and  repair.  The  replacement  transmitter 

2However,  when  the  F-16  radar’s  BIT  does  report  the  existence  of  a  fault,  the  report 
is  much  more  reliable  than  those  produced  by  the  F-15  radar’s  BIT. 
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appears  to  have  operated  as  it  should  for  the  remaining  five 
months  because  the  BIT  indicated  no  subsequent  faults  with  the 
transmitter. 

Although  the  transmitter  problem  was  fixed  during  June,  F-16 
Number  0577  had  other  problems  like  those  indicated  during  July  (see 
Fig.  4.2)  for  the  remaining  five  months  of  the  data  collection  effort. 
Throughout  the  program,  this  airplane  had  more  than  its  share  of  false 
target  and  target  lock-on  difficulties.  However,  these  problems  were 
rarely  brought  to  the  attention  of  maintenance  personnel.3  Moreover, 
the  radar  contractor  would  never  have  known  about  these  problems  if 
the  contractor’s  personnel  had  not  been  present  to  debrief  the  pilots. 

Part  of  the  problem  with  correcting  these  kinds  of  difficulties  is  that 
they  often  cannot  be  duplicated  on  the  ground.  Maintenance  techni¬ 
cians  will  therefore  often  assign  a  “CND”  to  the  problem  and  wait  to 
see  what  the  next  pilot  does.  The  data  show  that  the  next  pilot  often 
chooses  not  to  request  maintenance,  and  maintenance  personnel  there¬ 
fore  do  not  realize  that  the  radar  is  still  experiencing  problems. 

The  lack  of  a  continuing  record  on  how  individual  radars  perform 
creates  two  major  problems: 

•  Maintenance  resources  cannot  be  applied  to  the  radars  with  the 
greatest  need. 

•  Engineering  resources  cannot  be  applied  to  those  radar  R&M 
problems  that  cause  the  greatest  potential  degradation  in  perfor¬ 
mance. 

Figure  4.3  summarizes  data  for  all  150  airplanes  involved  in  the  spe¬ 
cial  data  collection  effort.  It  shows  that  when  the  objective  is  to  pro¬ 
vide  the  full  designed  capability  of  the  radar  on  a  regular  basis,  the 
standard  data  systems  portray  only  part  of  the  R&M  picture. 

The  two  standard  measures  most  frequently  cited  as  indicators  of 
R&M  status  are  the  fully  mission  capable  (FMC)  rate4  and  the  mean 
time  between  failure  (MTBF)  rate.5 


3A  major  factor  depressing  pilot  feedback  is  the  frequent  inability  of  maintenance 
technicians  to  isolate  the  cause  of  problems.  For  example,  half  of  pilot  requests  for 
maintenance  led  to  situations  where  the  flightline  maintenance  technicians  could  not 
find  any  fault  with  the  radar,  even  though  subsequent  analysis  revealed  that  nearly  all 
pilot  requests  were  well-founded. 

Calculated  from  data  collected  in  accordance  with  the  Air  Force’s  equipment  status 
reporting  procedures. 

Calculated  from  data  collected  in  accordance  with  the  Air  Force’s  maintenance  data 
collection  procedures.  Calculations  include  only  failures  that  are  determined  to  be 
relevant  and  confirmed  by  the  shop. 
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Mean  flight  hours 

Fig.  4.3 — Standard  F-16  A/B  data  show  partial  R&M  picture 


The  F-16’s  FMC  rate  is  well  over  80  percent,  an  impressive  record 
by  historical  standards  for  this  type  of  aircraft.  Moreover,  when  F-l6s 
are  not  FMC,  equipment  other  than  the  radar  mostly  accounts  for  this 
status.  However,  the  FMC  rate  is  an  incomplete  measure  because  it 
drops  only  when  pilots  request  flightline  maintenance,  and  it  rises 
again  as  soon  as  flightline  maintenance  is  completed.  For  avionics, 
flightline  maintenance  usually  can  be  completed  quickly  because  most 
such  actions  are  restricted  to  removing  and  replacing  an  LRU,  which  is 
then  sent  to  the  shop.  Removal  and  replacement  of  an  LRU  or  a  CND 
action  clears  a  Code  2  or  Code  3  discrepancy  but  often  does  not  correct 
the  fault.6 

The  F-16’s  MTBF  rate  similarly  causes  little  concern,  especially 
since  it  is  far  larger  than  the  MTBF  for  previous  radars.  However,  this 
measure  is  also  incomplete  because  it  reflects  only  some  situations 
where  an  LRU  fails  a  test  in  the  shop.  Not  all  shop  failures  were 
counted.  For  example,  when  the  flightline  sent  two  or  more  LRUs 
from  the  same  airplane,  only  one  LRU  failure  in  the  shop  was  counted 
when  the  MTBF  rate  was  calculated. 


6  About  one  half  of  the  time  such  actions  failed  to  correct  a  fault. 
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The  three  lower  bars  in  Fig.  4.3  represent  measures  based  on  data 
specially  collected  by  the  Radar  R&M  Program.  The  first  two  mea¬ 
sures  are  close:  (1)  the  99-hour  MTBF,  and  (2)  the  82  mean  flight 
hours  between  units  failing  a  shop  test,  as  they  should  since  MTBF  is 
one  way  of  looking  at  failures  from  the  shop’s  perspective.  But  when 
radar  R&M  is  viewed  from  the  standpoint  of  flightline  maintenance, 
there  are  just  31  hours  between  pilot  requests  for  maintenance.  And 
when  radar  R&M  is  viewed  from  the  standpoint  of  the  pilot,  there  are 
just  six  hours  between  indications  of  degraded  performance  in  the  cock¬ 
pit.' 

This  means  that  if  we  use  the  avionics  maintainability  indicator 
described  in  Sec.  Ill,  only  7  percent  of  the  F-16  radar’s  flights  with  one 
or  more  indicated  faults  led  to  a  fault  being  found  by  the  avionics  shop: 

Fault  6  MTBI 

Removal  =  -  x  100  =  7  percent 

Efficiency  82  MTBF 

But  even  the  six-hour  interval  between  flights  with  indicated  faults 
fails  to  portray  a  complete  picture  because  some  faults  are  not  indi¬ 
cated  during  every  flight.  Figure  4.4  shows  situations  when  the  BIT 
would  indicate  faults  during  some  flights  but  not  during  others.  Here, 
each  flight  with  an  LPRF  BIT  fault  indicates  that  the  radar  in  F-16 
number  0752  failed  a  BIT  that  clearly  identified  a  fault  in  the  LPRF 
LRU.  The  number  in  each  arrowhead  represents  the  number  of  con¬ 
secutive  flights.  Arrowheads  in  the  top  row  represent  flights  in  which 
the  BIT  did  not  indicate  a  fault  with  the  LPRF.  Arrowheads  in  the 
second  row  represent  flights  in  which  the  BIT  did  indicate  an  LPRF 
fault. 

Figure  4.4  shows  an  intermittent  pattern:  Two  consecutive  flights 
occur  with  faults,  one  without,  four  with,  one  without,  and  so  on.  This 
is  another  example  of  one  or  more  Type  B  faults.  Except  for  a  28- 
flight  respite  during  August,  this  pattern  persisted  throughout  the  six- 
month  data  collection  period.  The  28-flight  respite  occurred  after  a 
pilot  requested  maintenance  and  radar  technicians  replaced  the  LPRF. 
However,  even  during  these  28  flights,  the  pilot  observed  faults  that 


A  flight  with  an  indicated  fault  is  any  flight  in  which  the  pilot  reported  one  or  more 
fault  indications  to  a  contractor  data  collector  during  the  special  interview  following  the 
maintenance  debrief.  Fault  indications  included  situations  when  the  radar  failed  a  BIT 
as  well  as  when  the  pilot  believed  the  radar  operation  was  not  fulfilling  its  designed  capa¬ 
bility.  Some  of  the  pilot-reported  observations  of  difficulty  may  have  been  occasioned  by- 
increased  expectations  induced  by  the  presence  of  the  contractor's  data  collectors.  The 
resulting  effect  on  the  six  hours  must  be  small,  however,  because  80  percent  of  the  flights 
with  indications  had  one  or  more  faults  detected  by  the  BIT. 
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Fig.  4.4 — LPRF  history:  F-16  number  0752 
(June  through  December  1984) 


the  BIT  eventually  picked  up,  so  it  is  reasonable  to  assume  that  one  or 
more  faults  occurred  in  the  replacement  LPRF.  Yet  throughout  the  six 
months  there  were  only  two  requests  for  maintenance  and  only  one 
LRU  removed.8 

It  is  also  reasonable  to  believe  that  during  the  other  89  flights  before 
and  after  that  28-flight  period  one  or  more  faults  occurred  in  the  LPRF 
that  could  inhibit  the  radar  from  delivering  its  full  designed  capability. 
Thus,  although  only  41  flights  yielded  BIT  indications  of  faults,  twice 
that  number  of  flights  were  apparently  flown  with  one  or  more  faults 
present  in  the  LPRF. 

Finally,  Fig.  4.5  summarizes  the  LPRF  BIT  failures  for  the  10th 
Tactical  Fighter  Squadron  (representative  of  the  six  squadrons 

HMoreover,  Westinghouse  tracked  this  airplane  for  an  extra  month,  until  the  Air 
Force— at  Westinghouse’s  suggestion— removed  and  repaired  this  LRU. 
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monitored)  for  an  entire  six-month  period.  The  left  column  identifies 
the  specific  aircraft.  Unshaded  portions  of  the  bars  represent  series  of 
flights  during  which  the  BIT  indicated  the  LPRF  had  functioned  prop¬ 
erly.  Shaded  portions  represent  series  of  flights  during  which  the  BIT 
indicated  the  LPRF  had  failed  to  function  properly. 

Fortunately,  not  all  airplanes  look  like  0752.  For  example,  the  BIT 
in  number  0556  found  no  faults  in  the  LPRF  for  the  entire  four-month 
period  that  it  participated  in  the  data  collection  program.  Altogether, 
12  airplanes  in  this  squadron  had  no  faults  in  their  LPRFs  as  indicated 
by  their  BITs.  However,  ten  airplanes  encountered  protracted  periods 
spanning  several  months  during  which  they  had  faults  in  their  LPRFs. 
(The  fact  that  so  many  aircraft  are  afflicted  with  LPRF  faults  had  not 
been  documented  before  the  data  collection  effort.) 

Candidate  Improvements.  Figure  4.6  shows  the  current  LPRF 
unit,  which  is  the  primary  source  of  radar  R&M  problems  for  the  F-16 


Fig.  4.5 — LPRF  BIT  failure  episodes:  10th  TFS 
(June  through  December  1984) 


48 


radar.  Although  critical  to  the  F-16’s  wartime  mission,  this  LPRF  has 
never  adequately  “matured.” 

As  with  the  radar  and  other  modern  avionics  on  the  F-16,  this  unit 
rarely  experiences  a  failure  severe  enough  to  render  it  totally  incapable 
of  supporting  the  airplane’s  missions.  Its  most  common  failure  modes 
have  two  characteristics: 

•  They  degrade  the  available  level  of  performance  by  reducing 
detection  ranges,  introducing  target  tracking  problems,  or  creat¬ 
ing  false  targets. 

•  They  manifest  themselves  mostly  when  the  radar  is  exposed  to 
thermal  and  dynamic  stresses  from  the  airplane’s  operational 
environment.  Thus  the  BIT  will  frequently  catch  a  fault  during 
flight  but  later  fail  to  duplicate  it  when  the  airplane  is  on  the 
ground. 

The  dominant  causes  of  such  performance  degradations  are  a  material 
incompatibility  problem  in  the  receiver  module  and  a  packaging  of  the 
LRU  that  complicates  reassembly  of  the  unit  after  maintenance.  The 
material’s  problem  is  worsening  with  age  because  a  chemical  process  is 
causing  deterioration  of  delicate  circuits.  The  reassembly  problem  is 
caused  by  a  packaging  design  that  requires  extensive  disassembly  to 
gain  access  to  faulty  components.  The  reassembly  process  requires  the 
technician  to  connect  and  align  several  delicate  rf  fittings  that  are  hard 
to  access.  (The  more  easily  accessed  fittings  are  visible  in  Fig.  4.6.) 

The  Radar  R&M  Program  identified  three  causes  of  problems  with 
the  LPRF: 

1.  Connections  between  modules.  Delicate  radio  frequency  connec¬ 
tions  between  modules  have  stringent  alignment  and  torque 
requirements.  Technicians  must  undo  many  of  these  connec¬ 
tions  to  get  at  modules  and  circuit  cards  inside  the  LPRF. 
Imprecise  refastening  can  let  the  dynamic  flight  environment 
disrupt  signals  and  thereby  lead  to  false  targets  and  trigger  BIT 
failures;  clearance  limitations  in  the  present  design  make  precise 
refastening  very  difficult  to  achieve. 

2.  Components  within  modules.  Several  components  within 
modules  fail  mostly  when  subjected  to  thermal  and  flight 
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Fig.  4.6 — Current  LPRF  for  the  F-16 


dynamic  stresses.  Thus  the  BIT  often  finds  faults  in  flight  that 
neither  it  nor  the  shop  can  find  on  the  ground.9 

3.  Connections  between  components  within  modules.  Deteriora¬ 
tion  of  connections  within  the  receiver  module  is  occurring 
throughout  the  module  where  indium  solder  joins  gold  conductor 
ribbons.  The  indium  solder  absorbs  molecules  in  the  connector 
ribbon,  and  as  these  molecules  migrate  they  leave  voids  in  the 
conductor  ribbon  that  coalesce  to  form  minute  flaws.  When 
exposed  to  the  cyclic  stresses  created  by  the  time-varying  ther¬ 
mal  and  dynamic  environment,  these  flaws  enlarge  much  like  a 
fatigue  crack  would  in  an  airframe  structure.  Eventually,  the 
connector  ribbon  breaks  away  from  the  solder  and  causes  an 
open  circuit  that  the  shop’s  test  equipment  can  readily  detect. 
Long  before  separation,  however,  the  quality  of  the  radio 

9Both  shops  and  depots  test  the  equipment  at  normal  room  temperatures  and  in  a 
dynamically  benign  environment  rather  than  under  simulated  flight  environment  condi¬ 
tions. 
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frequency  circuitry  begins  to  degrade,  and  such  degradation  can 
manifest  itself  in  the  form  of  BIT-detected  faults,  false  targets, 
tracking  problems,  and  the  like. 


These  problems  with  the  LPRF  are  worsening  with  age,  as  evidenced 
by  a  60  percent  increase  in  monthly  receiver  repairs  from  1984  to  1985. 

Before  the  special  data  collection  and  analysis  effort,  problems  with 
the  LPRF  were  masked  by  the  confluence  of  several  factors.  For  one 
thing,  a  radar  with  a  99-hour  MTBF  does  not  draw  much  attention,10 
especially  when  it  was  not  known  that  there  were  only  six  hours 
between  flights  with  indicated  faults  (and  80  percent  of  these  indica¬ 
tions  were  by  the  BIT).  Moreover,  false  targets,  target  tracking  prob¬ 
lems,  and  the  like  can  often  result  from  factors  external  to  the  aircraft. 
In  addition,  the  extent  of  environment-sensitive  failure  modes  was 
unknown.  Before  the  special  data  collection,  there  were  no  records 
that  could  pinpoint  indicated  fault  trends  to  system-specific  problems. 
Finally,  many  faults  detected  by  the  BIT  were  thought  to  result  from  a 
grounding  problem  that  could  be  corrected  by  a  new  shim  that  would 
provide  a  tighter  electrical  path  for  grounding  the  radar  equipment  to 
the  airframe.11 

In  part,  problems  with  the  LPRF  resulted  from  squeezing  a  capable 
and  versatile  radar  into  a  very  small  volume  with  a  tight  weight  limita¬ 
tion  and  a  limited  amount  of  environmental  cooling.  Advances  in  tech¬ 
nology  will  now  permit  a  more  maintainable  LPRF  that  should  be  less 
prone  to  the  kinds  of  connection  and  component  problems  that  plague 
the  current  design. 

To  explore  this  idea,  the  Radar  R&M  Program  commissioned  West- 
inghouse  to  develop  a  preliminary  design  for  a  new  LPRF  that  would 
draw  on  technologies  that  Westinghouse  was  already  incorporating  in 
its  newer  radars  (see  Fig.  4.7).  This  new  design  resolves  the  receiver 
problem  and  enables  more  circuitry  to  be  packaged  into  a  smaller 
volume.  Such  packaging  is  also  more  modular  and  facilitates  disassem¬ 
bly.  With  connections  on  the  back  of  each  module,  shop  technicians 
would  no  longer  have  to  disrupt  as  much  of  the  delicate  radio  frequency 
circuitry  to  remove  a  module  such  as  a  redesigned  receiver. 

R&M  Improvement  Package.  Redesign  of  the  LPRF  would  serve 
as  the  cornerstone  in  implementing  maturational  development 
improvements  for  the  F-16's  radar.  A  complete  package  would  include 

l0The  F-4’8  radar  has  less  than  a  9-hour  MTBF. 

"Special  tests  showed  that  the  new  shim  would  not  correct  problems  the  BIT  was 
attributing  to  the  LPRF:  Of  20  aircraft  that  received  new  shims,  19  were  found  to  have 
faulty  LPRF  units  that  caused  the  BIT  fault  indications. 
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the  items  listed  in  Table  4.1  and  would  fall  into  three  priority  classifi¬ 
cations  based  on  subjective  assessments  of  their  potential  benefits.12 
Priority  2  improvements  for  the  F-16  radar  consist  of 

1.  Redesigning  and  re-routing  the  antenna  cable  to  allow  replace¬ 
ment  at  the  avionics  shop  and  thus  avoid  having  to  send  the 
entire  antenna  to  the  depot  for  cable  replacement. 

2.  Using  recently  developed  technologies  to  improve  the  maintain¬ 
ability  of  the  radio  frequency  test  station,  currently  the  most  dif¬ 
ficult  station  to  maintain  in  the  avionics  shop. 

3.  Instituting  environmental  testing  at  the  depot  to  improve  repair 
of  the  current  LPRF  and  other  bad-actor  equipment.13 

Priority  3  improvements  for  the  F-16  consist  of 

1.  Changing  Technical  Orders  to  make  them  more  consistent,  com¬ 
pact,  accurate,  and  useful  for  maintenance  personnel. 

12In  spite  of  these  classifications,  we  recommend  that  the  Air  Force  pursue  all  the 
identified  improvements. 

I3During  the  data  collection  and  analysis  program,  30  suspected  bad-actor  LRUs  were 
sent  to  the  contractor.  By  just  applying  cold  soak  to  these  LRUs  and  then  testing  them, 
the  contractors  were  able  to  isolate  the  faulty  SRUs  in  two-thirds  (20)  of  the  LRUs.  By 
adding  a  vibration  environment  to  the  test  they  may  have  been  able  to  isolate  the 
remaining  LRUs’  faults. 


52 


2.  Correcting  deficiencies  in  shop  software  that  allow  certain  faults 
to  escape  detection. 

(The  “generic  improvements”  listed  in  Table  4.1  are  described  at  the 
end  of  this  section.) 

Such  a  total  improvement  package  could  triple  the  mean  time 
between  indicated  faults  (from  six  hours  to  19  hours)  and  also  reduce 
the  maintenance  workload  by  35  percent.  In  total,  this  development 
package  (excluding  Priority  3  items)  would  cost  $250  million.14 
Although  some  of  this  cost  could  be  offset  by  the  decreased  need  for 
maintenance,  the  main  benefit  would  be  in  increased  availability  of 
radars  that  are  prepared  to  dependably  deliver  the  performance  they 
were  designed  for. 


Table  4.1 

A  MATURATIONAL  DEVELOPMENT  IMPLEMENTATION  PACKAGE 
FOR  THE  F-16’S  RADAR 


Priority 

1 

Priority 

2 

Priority 

3 

F-16 

Specific 

LPRF 

Antenna  cable 

Radio  frequency 
test  station 

Depot  environmental 
testing  for  LPRF 

Technical  Orders 
Shop  software 

Generic 

Tracking  equipment 
by  serial  number 
Test  translation 
dictionaries 

Direct  entry 
testing 

Loop  testing 

Material  deficiency 
reporting 
Technical  Orders 
feedback 
Interactive  pilot 
debrief 

^Quantitative  estimates  for  costs  and  benefits  were  derived  from  the  technical  data 
packages  that  Westinghouse  delivered  to  the  Strike  SPO.  We  reviewed  the  data  pack¬ 
ages  as  did  a  Technical  Coordinating  Committee  that  included  R&M  engineers  from  the 
Air  Force  Acquisition  Logistics  Center,  an  independent  contractor  (Support  Systems 
Associates,  Inc.),  and  personnel  from  the  strike  SPO.  The  independent  contractor  and 
the  RAND  personnel  monitored  both  the  data  collection  and  the  engineering  analysis 
efforts. 
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Stage  2:  Implementation  of  Improvements 

At  present  we  can  report  the  current  status  of  three  recommended 
R&M  improvements  for  the  F-16  A/B  radar: 

•  The  design  effort  on  a  new  LPRF  (Priority  1)  is  scheduled  to 
start  in  1989. 

•  The  new  design  for  an  antenna  cable  (Priority  2)  has  completed 
development. 

•  Changes  to  TOs  (Priority  3)  have  been  submitted. 


RESULTS  SPECIFIC  TO  THE  F-15  RADAR 

Stage  I:  Assessment  of  R&M  Situation 

For  this  radar,  the  Strike  SPO  contracted  with  the  Hughes  Aircraft 
Company  for  the  four  major  activities  constituting  Stage  1  of  matura- 
tional  development.  The  one-year  contract  for  approximately  $6  million 
contained  provisions  under  which  Hughes  contracted  with  McDonnell 
Douglas  (the  F-15  weapon  system  prime  contractor)  for  engineering  assis¬ 
tance. 

Data  Collection.  Data  on  the  F-15’s  radar  were  collected  from  150 
F-15s  monitored  during  a  six-month  period  (June-December  1984)  at 
Langley  AFB  and  Bitburg  Air  Base.  These  data  covered  16,702  flights 
and  the  resulting  radar  maintenance  at  these  bases.  As  with  the  F-16’s 
radar,  contractor  personnel  interviewed  pilots  after  every  flight  and 
then  documented  maintenance  on  the  flightline,  in  the  shop,  and  at  the 
depot.  This  effort  required  the  fielding  of  a  data  collection  team  that 
included  32  people  provided  by  the  radar  contractor  (Hughes)  and  four 
people  provided  by  the  weapon  system  prime  contractor  (McDonnell 
Aircraft). 

Analysis.  The  radar  for  the  F-15  C/D  is  more  powerful,  has  a 
larger  antenna,  and  is  heavier  and  more  complicated  than  the  F-16 
radar.  It  also  has  more  capability  even  though  its  technology  precedes 
the  F-16’s  radar  by  about  four  years.  As  Fig.  4.8  shows,  the  F-15  radar 
has  nine  LRUs  in  addition  to  the  throttle,  stick,  and  connectors.  This 
number  of  LRUs  creates  problems  because  the  BIT  on  this  radar, 
unlike  that  on  the  F-16,  is  not  dependable  at  identifying  the  LRU  that 
the  technician  should  remove. 

For  the  F-15  radar,  the  main  R&M  problems  arise  with  the  BIT 
itself.  The  F-15’s  BIT  aims  at  detecting  faults  in  the  radar  and  at  iso¬ 
lating  the  causes  to  a  particular  LRU.  Neither  it  nor  other  BITs,  how¬ 
ever,  isolate  faults  in  cables  and  connectors.  The  BIT  operates  during 


flight  in  a  mode  called  “continuous  monitor”  in  which  it  provides  short, 
periodic  tests.  In  addition,  the  operator  can  use  it  on  the  ground  or  in 
flight  to  initiate  comprehensive,  three-minute  tests. 

The  extent  and  severity  of  the  R&M  problems  with  the  BIT  were 
determined  and  validated  by  a  three -step  process  that  included  special 
data  collection,  engineering  analysis,  and  independent  review.  Figure 
4.9  summarizes  data  for  all  150  airplanes  involved  in  the  special  data 
collection  effort. 

The  F-15’s  FMC  rate  is  well  over  70  percent  and  thus  might  suggest 
no  serious  R&M  problems.  Moreover,  Fig.  4.9  shows  the  F-15’s  radar 
MTBF  rate  is  23  hours,  much  lower  than  the  99  for  the  F-16’s  radar 
but  also  much  higher  than  the  single-digit  MTBF  for  the  F-4’s  radar. 
This  difference  should  not  be  too  surprising  because  the  F-15’s  radar  is 
more  complex  and  represents  an  older  technology  than  the  F-16’s 
radar. 

The  three  lower  bars  of  Fig.  4.9  represent  measures  based  on  the 
Radar  R&M  program  data.  Again,  the  first  two  values  are  close:  (1) 
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Fig.  4.9 — Standard  F-15  C/D  data  show  partial  R&M  picture 


the  23-hour  MTBF,  and  (2)  the  19  mean  flight  hours  between  units 
failing  a  shop  test,  as  MTBF  is  one  way  of  looking  at  failures  from  the 
shop’s  perspective. 

But  when  radar  R&M  is  viewed  from  the  standpoint  of  flightline 
maintenance  and  its  problems  of  dealing  with  the  faulty  BIT,  there  are 
just  eight  hours  between  pilots’  requests  for  maintenance.  And  when 
radar  R&M  is  viewed  from  the  standpoint  of  the  pilots  and  their  prob¬ 
lems  with  both  the  radar  and  the  faulty  BIT,  there  are  just  four  hours 
between  indications  in  the  cockpit  of  degraded  performance. 

This  means  that  if  we  use  the  avionics  maintainability  indicator 
described  in  Sec.  Ill,  only  21  percent  of  the  F-15  radar’s  flights  with 
one  or  more  indicated  faults  led  to  a  fault’s  being  found  by  the  avionics 
shop: 

Fault  4  MTBI 

Removal  =  -  x  100  =  21  percent 

Efficiency  19  MTBF 

Indeed,  28  percent  of  the  time  that  a  pilot  saw  a  problem  with  the 
radar,  the  BIT  failed  to  indicate  a  fault.  According  to  Hughes’s 
engineering  analysis  of  their  collected  data,  up  to  35  percent  of  the 
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time  that  the  BIT  indicates  a  failure,  the  indication  is  wrong.  And 
when  the  BIT  detects  a  real  failure,  up  to  35  percent  of  the  time  the 
BIT  isolates  the  fault  to  the  wrong  LRU,  which  causes  maintenance 
technicians  to  look  in  the  wrong  place.  The  low  dependability  of  the 
BIT  means  that  problems  can  linger  in  the  aircraft  for  long  periods  of 
time  before  they  are  finally  located  and  solved. 

A  faulty  BIT  means  several  things.  Sometimes  the  BIT  indicates  a 
fault  when  there  is  none.  At  other  times  the  BIT  does  not  indicate  an 
existing  fault.  All  of  this  leads  pilots  and  maintenance  personnel  to 
distrust  the  BIT.  This  distrust  has,  among  other  things,  led  mainte¬ 
nance  personnel  to  swap  LRUs  between  airplanes  to  isolate  faulty 
units.  But  this  kind  of  inefficiency,  necessitated  by  the  low  depend¬ 
ability  of  the  BIT,  greatly  increases  the  maintenance  workload.15 

Figure  4.10  documents  this  inefficiency  by  showing  that  mainte¬ 
nance  personnel  have  to  remove  almost  twice  as  many  LRUs  as  actu¬ 
ally  have  problems.  Each  of  the  clear  bars  represents  the  number  of 
occurrences  for  the  events  defined  in  the  left  margin.  If  the  mainte¬ 
nance  process  perfectly  followed  the  underlying  maintenance  concept, 
all  of  the  clear  bars  would  have  the  same  length.  That  is,  for  each 
maintenance  request,  one  problem  would  be  found,  one  LRU  would  be 
removed,  and  one  LRU  would  be  sent  to  the  shop.  The  shaded  bars 
explain  deviations  from  this  ideal.  For  example,  the  “Maintenance 
Requested”  bar  greatly  exceeds  the  “Problem(s)  Found”  bar.  The  rea¬ 
son  lies  in  the  shaded  “Can  Not  Duplicate”  bar.  These  were  events 
when  flightline  maintenance  personnel  could  not  duplicate  symptoms 
for  the  problems  for  which  the  pilot  had  requested  maintenance. 

The  next  major  deviation  occurs  between  the  “Problem(s)  Found” 
bar  and  the  “LRU  Removed”  bar.  Far  more  LRUs  are  being  removed 
than  there  are  problems.  In  fact,  the  number  of  extra  LRU  removals  is 
even  greater  than  the  difference  in  the  clear  bars  because  some  of  the 
problems  are  dealt  with  by  fixing  a  connection  problem  that  lies 
between  LRUs — no  LRU  is  removed.  There  were  nearly  two  LRUs 
removed  for  every  problem  that  required  an  LRU  removal.  Many  of 
these  extra  removals  were  LRUs  that  were  moved  between  aircraft  to 
facilitate  fault  isolation.  Some,  however,  were  sent  to  the  shop  (this  is 
the  difference  between  the  last  two  shaded  bars).  In  fact,  the  shop 
received  about  one-fourth  more  LRUs  than  there  were  problems. 

In  sum,  BIT  deficiencies  increase  the  maintenance  burden  by  nearly 
doubling  flightline  removals  and  by  increasing  shop  arrivals  by  one- 
fourth. 

15It  also  wears  the  connectors  and  wires  that  join  LRUs  together.  Connector  and  wir¬ 
ing  problems  are  especially  hard  to  isolate. 
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Fig.  4.10 — Flightline  maintenance  events:  F-15  radar 
(June  through  December  1984) 


Candidate  Improvements.  To  deal  with  the  F-15’s  BIT  problems, 
the  Radar  R&M  Program  identified  three  needed  improvements: 

1.  Improvements  to  the  BIT  software.  Although  new  tactical 
software  is  developed  and  tested  each  year,  the  BIT  has  not 
been  retested.  During  the  Radar  R&M  Program,  Hughes 
engineers  found  ten  situations  in  which  the  BIT,  because  of  the 
way  that  it  is  interleaved  with  the  tactical  software,  falsely  indi¬ 
cates  the  presence  of  faults. 

2.  Additions  to  the  BIT  hardware.  During  the  data  collection 
effort,  Warner  Robins  ALC  was  working  on  ground  support 
equipment  that  could  augment  the  BIT  function.  Since  this 
equipment  necessarily  requires  up  to  two  or  three  hours  for  use, 
it  was  not  designed  to  support  the  quick  turnarounds  required  in 
wartime.  It  could,  however,  be  available  within  about  two  years. 

3.  Improvements  to  the  BIT  hardware.  Adding  memory  and  test 
points  to  the  BIT  hardware  would  provide  more  of  the  capabili¬ 
ties  that  are  being  incorporated  in  the  ground  support 
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equipment  and  would  allow  quick  turnarounds  without  more 
ground  support  equipment. 

Since  the  F-15  radar  first  entered  operational  service  in  1975,  it  is 
reasonable  to  enquire  why  the  BIT  deficiencies  had  not  been  resolved 
much  sooner.  A  major  factor  is  that  the  radar  R&M  program  provided 
the  radar  contractor  the  first  opportunity  to  make  an  in-depth,  first¬ 
hand  engineering  examination  of  what  was  happening  in  the  field. 
Although  the  radar  contractor  had  had  technical  representatives 
assigned  to  various  bases  over  the  years,  these  representatives  could 
not  begin  to  collect  the  kinds  of  data  needed  to  define  the  specific 
engineering  deficiencies  with  the  current  BIT. 

Improvement  Package.  Improvements  to  the  BIT’s  software  and 
hardware  would  serve  as  the  cornerstone  in  implementing  maturational 
development  improvements  for  the  F-15’s  radar.  A  complete  package 
of  these  improvements  would  include  the  items  listed  in  Table  4.2. 
The  improvements  fall  into  three  priority  classifications  based  on  sub¬ 
jective  assessments  of  their  potential  benefits.16  Priority  2  improve¬ 
ments  for  the  F-15  consist  of 

1.  Undertaking  LRU  fixes  to  improve  the  reliability  for  the  radar’s 
exciter,  transmitter,  receiver,  antenna,  analog  processor,  pro¬ 
grammable  signal  processor,  and  power  supply. 

2.  Adding  dynamic  test  and  fault-isolation  capabilities  for  the 
Antenna  A  Test  Station  in  the  avionics  shop. 

3.  Instituting  environmental  testing  and  adding  a  complete  radar 
test  bench  to  the  depot  equipment.1 ' 

4.  Undertaking  research,  evaluation,  and  selection  of  a  replacement 
for  coolanol,  which  is  prone  to  contamination. 

Priority  3  improvements  for  the  F-15  consist  of  specific  changes  pro¬ 
posed  for 

1.  Changing  Technical  Orders  to  make  them  more  consistent,  com¬ 
pact,  accurate,  and  useful  for  maintenance  personnel. 

2.  Correcting  deficiencies  in  shop  software  that  allow  certain  faults 
to  escape  detection. 

3.  Identifying  reasons  why  problems  with  SRU  connections 
apparently  disappear  when  maintenance  personnel  reseat  the 
SRU  in  the  LRU. 

16In  spite  of  these  classifications,  we  recommend  that  the  Air  Force  pursue  all  the 
identified  improvements. 

17The  test  bench  allows  technicians  to  test  any  radar  LRU  as  part  of  the  overall  radar 
subsystem. 
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Such  a  total  improvement  package  could  triple  the  MTBI  faults 
(from  four  hours  to  13  hours)  and  reduce  the  maintenance  workload  by 
50  percent.  In  total,  this  development  package  (excluding  Priority  3 
items)  would  cost  $200  million.18  Although  some  of  this  cost  could  be 
offset  by  the  decreased  need  for  maintenance,  the  main  benefit  would 
be  in  the  increased  availability  of  radars  that  are  prepared  to  depend¬ 
ably  deliver  the  complete  performance  they  were  designed  for. 


Stage  2:  Implementation  of  Improvements 

At  present,  we  can  report  the  current  status  of  five  recommended 
R&M  improvements  for  the  F-15  C/D  Radar: 

•  Development  of  the  BIT  Software  (Priority  1)  is  underway  and 
flight  testing  of  the  new  software  was  scheduled  for  September 
1987. 

•  LRU  reliability  fixes  and  depot  equipment  (both  Priority  2)  are 
underway. 

•  The  study  of  a  coolant  replacement  (Priority  2)  is  in  progress. 

•  Changes  to  TOs  (Priority  3)  have  been  submitted. 

•  Study  of  the  reseat  problems  (Priority  3)  is  underway. 

Table  4.2 

A  MATURATIONAL  DEVELOPMENT  IMPLEMENTATION  PACKAGE 
FOR  THE  F-15'S  RADAR 


Priority 

1 

Priority 

2 

Priority 

3 

F-15 

BIT  software 

LRU  fixes 

Technical  Orders 

Specific 

and  hardware 

Antenna  A  Test 
Station 

Depot  equipment 
Replacement  of 
coolanol 

Shop  software 
Reseat  problems 

Generic 

Tracking  equipment 
by  serial  number 
Test  translation 
dictionaries 

Direct  entry 
testing 

Loop  testing 

Material  deficient 
reporting 
Technical  Orders 
feedback 
Interactive  pilot 
debrief 

'"Quantitative  estimates  for  costs  and  benefits  were  derived  from  the  technical  data 
packages  that  Hughes  delivered  to  the  Strike  SPO.  The  same  Technical  Coordinating 
Committee  that  reviewed  the  Westinghouse  lata  packages  reviewed  these  as  well 
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RESULTS  COMMON  TO  BOTH  RADARS 

Stage  1:  Assessment  of  R&M  Situation 

Although  these  exploratory  applications  of  Stage  1  of  maturational 
development  concentrated  on  radars  for  the  F-16  and  F-15,  the  com¬ 
mon  results  are  potentially  applicable  to  other  types  of  similarly 
sophisticated  airborne  electronics,  such  as  weapon  delivery  and  elec¬ 
tronic  warfare  equipment,  and  to  other  combat  aircraft,  such  as  attack 
aircraft  and  bombers. 

The  Stage  1  results  for  both  radars  showed  faults  escaping  detection 
in  the  cockpit,  on  the  flightline,  in  the  avion. „s  shop,  and  at  the  depot. 
Because  testing  was  found  to  be  far  from  infallible  at  every  level  of 
maintenance,  and  because  faults  that  were  correctly  detected  were 
sometimes  not  correctly  fixed,  many  faulty  units  circulated  through  the 
system  and  back  into  the  airplanes.  Such  units  left  a  history  of 
degraded  performance  that  the  standard  data  collection  system  did  not 
detect. 

This  circulation  of  faulty  units  through  the  system  created  three 
major  problems.  First,  airplanes  with  faulty  units  could  not  depend¬ 
ably  deliver  their  full  range  of  designed  capabilities.  This  situation 
introduces  an  intolerable  uncertainty  into  the  combat  arena.  Second, 
pilots  often  could  not  recognize  these  faults  because  of  the  multi¬ 
capability  nature  of  their  avionics  equipment.  Third,  faulty  units 
sooner  or  later  displayed  degraded  performance,  and  maintenance  tech¬ 
nicians  again  had  to  deal  with  them.  When  faulty  units  make  multiple 
passes  through  maintenance,  the  maintenance  workload  is  artificially 
increased — an  all  too  frequent  occurrence  with  the  F-15  and  F-16 
radars. 

Taken  in  conjunction  with  maturing  of  the  F-16  and  F-15  radars, 
seven  generic  changes  applicable  to  the  support  process  for  both  radars 
can  greatly  improve  the  R&M  situation.  These  improvements  fall  into 
three  priority  classifications  based  on  subjective  assessments  of  their 
potential  benefits:19 

Priority  1.  Tracking  equipment  by  serial  number,  and  test  transla¬ 
tion  dictionaries. 


l9In  spite  of  these  claf  '.cations,  we  recommend  that  the  Air  Force  pursue  all  the 
identified  improvements. 


Priority  2.  Direct  entry  into  test  sequences,  and  loop  testing. 

Priority  3.  Material  deficiency  reporting,  TO  feedback,  and  interac¬ 
tive  pilot  debriefing. 

Priority  1  improvements  aim  at  more  rapid  identification  of  faulty 
units. 

Tracking  Equipment  by  Serial  Number.  Table  4.3  identifies  by 
serial  number  the  specific  F-15  radar  LRUs  that  made  multiple  visits 
to  the  shop  during  the  six-month  special  data  collection  period.  Tech¬ 
nicians  use  the  term  “bad  actors”  for  repeatedly  faulty  LRUs  that  the 
shop  cannot  correct.  Programmable  Signal  Processor  #1059  made  a 
dozen  visits  to  the  shop  in  six  months  when  one  would  expect  to  see  it 
in  the  shop  no  more  than  once.  Perhaps  the  symptoms  of  the  fault(s) 
with  this  bad  actor  were  stimulated  only  in  the  flight  environment, 
which  the  shop  could  not  duplicate,  or  perhaps  the  shop’s  testing  pro¬ 
cedures  failed  to  detect  the  problem  because  of  a  void  in  the  test  logic. 

Effective  tracking  of  equipment  by  serial  number  must  begin  with  a 
full  debriefing  of  the  pilot  during  which  all  indications  of  faults  are 
reported  to  maintenance.  It  then  involves  record  keeping  by  mainte¬ 
nance  personnel  on  the  flightline,  in  the  shop,  and  at  the  depot  con¬ 
cerning  the  status  of  equipment  taken  from  the  airplane,  tested,  and 
repaired. 

Table  4.3 

F-15  RADAR  LRUs  WITH  MULTIPLE 
VISITS  TO  THE  SHOP3 

Visits  per 


Unit  Type  and  Serial  Number  Unit 

Programmable  signal  processor  =1059  12 

Transmitter  =0067  10 

Programmable  signal  processor  =1015  9 

Transmitter  =0525  8 

Receiver  =0471  8 

Analog  signal  processor  =0561  8 

Programmable  signal  processor  =  1057  8 

Antenna  =0855  7 

Seven  various  units  6 

Seventeen  various  units  5 


“Data  collected  at  Bit  burg  Air  Base.  -June  to 
December  1981. 
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This  kind  of  tracking  of  equipment  by  serial  number  has  long  been  a 
contentious  subject.  The  logistics  community  has  not  yet  universally 
accepted  this  need.  As  some  logisticians  see  it,  the  development  com¬ 
munity  should  solve  the  problem  by  improved  designs  for  LRUs; 
maintenance  personnel  should  not  have  to  keep  detailed  historical 
records  because  their  primary  job  is  to  be  prepared  to  fix  airplanes 
quickly.  Although  placing  greater  burdens  on  the  developers  may  help 
upcoming  systems,  it  will  not  help  currently  fielded  weapon  systems. 
For  these  systems,  tracking  equipment  by  serial  number  can  help  iden¬ 
tify  specific  bad  actors  that  waste  maintenance  resources  and  under¬ 
mine  fighter  effectiveness. 

We  believe  that  this  kind  of  improved  information  is  essential  to 
improved  training  of  avionics  maintenance  personnel.  On  a  few  occa¬ 
sions  we  have  had  the  opportunity  to  compare  the  performances  of 
contractor  maintenance  personnel  with  those  of  less  well-trained  and 
less  experienced  Air  Force  maintenance  personnel;  we  have  seen  no 
difference  in  the  effectiveness  of  Air  Force  personnel  performances  as 
measured  in  terms  of  mean  flight  time  between  LRU  removals  on  the 
flightline,  mean  flight  time  between  LRU  failures  verified  by  the  shop, 
and  percent  of  LRUs  that  bench  check  serviceable  (BCS)  in  the  shop 
(see  Table  4.4).20  These  two  groups  have  the  same  record  in  dealing 
with  faulty  LRUs  largely  because  they  must  deal  with  the  same  kinds 
of  limited  information. 

Test  Translation  Dictionaries.  Faulty  units  can  often  slip 
undetected  through  several  layers  of  maintenance  because  testing 
varies  from  level  to  level.  Tests  in  the  airplane  are  different  from  tests 
in  the  shop,  and  tests  in  the  shop  are  often  different  from  tests  in  the 
depot.  LRUs  at  the  depot  receive  the  same  tests  as  those  in  the  shop, 
but  SRUs  at  the  depot  undergo  more  detailed  testing.  Unfortunately, 
tests  at  these  different  levels  speak  “different  languages.”  Thus  techni¬ 
cians  at  one  level  often  cannot  benefit  from  the  testing  done  at  another 
level. 

“Translation  dictionaries”  could  enable  technicians  at  one  level  to 
translate  test  results  from  some  other  level  into  terms  they  can  find 
useful.  Technicians  must  use  information  from  the  previous  level  to 
confirm  that  they  are  correcting  failures  discovered  the  previous 
level.  If  test  results  at  a  more  detailed  level  do  not  verify  results  from 
the  preceding  level,  the  unit  may  be  a  bad  actor  and  require  special 
surveillance  by  pilots  and  flightline  maintenance  personnel.  And  if  the 

20A  BCS  occurs  when  an  LRU  received  by  the  shop  passes  all  shop  tests  without 
maintenance  being  performed.  Typically,  ,'!5  percent  of  complex  avionics  LRUs  bench 
check  serviceable  in  the  shop. 


Table  4.4 


EFFECTS  OF  TRAINING  AND  EXPERIENCE  ON 
MAINTENANCE  PERFORMANCE® 


Measures  of 

Maintenance  Performance 

Contractor 

Maintenance 

Personnel 

Air  Force 
Maintenance 
Personnel 

Mean  flight  time 
between  avionics 

LRU  removals  on 
the  flightline 

2.1  hrs 

2.0  hrs 

LRU  failures  verified 
by  the  shop 

3.0  hrs 

2.9  hrs 

Percent  of  avionics  LRUs  that 
bench  checked  serviceable 

30.2 

27.5 

“Data  was  collected  as  part  of  the  F-111D  MK  II  Fault 
Isolation  Verification  Program.  Data  on  General  Dynamics 
maintenance  personnel  cover  179.4  flight  hours  between 
December  12,  1973,  and  March  21,  1974;  data  on  Air  Force 
maintenance  personnel  cover  14,667  flight  hours  between 
January  1,  1973,  and  December  31,  1973.  This  information 
was  extracted  from  the  personnel  files  of  H.  L.  Shulman, 
who  was  instrumental  in  initiating  and  directing  the  pro¬ 
gram. 


unit’s  anomalous  behavior  persists,  it  needs  to  be  sent  to  the  depot  for 
special  testing  and  analysis. 

During  the  radar  R&M  program,  radar  contractors  were  able  to 
identify  and  fix  bad  actors  because  their  engineers  could  track  equip¬ 
ment  by  serial  number  and  perform  needed  translations  of  test  results. 
Moreover,  there  were  two  very  important  and  interesting  discoveries: 

•  In  the  case  of  the  F-16  radar,  the  LPRF  accounted  for  most  of 
the  LRUs  entering  the  shop.  But  half  of  these  LRUs  would 
bench  check  serviceable.  Tracking  specific  units  by  serial 
number  indicated  that  most  of  these  units  were  indeed  faulty. 

•  In  the  case  of  the  F-15  radar,  about  35  percent  of  the  LRUs 
entering  the  shop  would  bench  check  serviceable.  Moreover,  for 
an  additional  20  percent  that  received  maintenance  actions,  those 
actions  were  unrelated  to  real  inflight  faults.  Less  than  half  of 
the  LRUs  entering  the  shop  had  maintenance  performed  that  was 
relevant  to  inflight  faults. 
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Priority  2  improvements  aim  at  increasing  shop  and  depot  capability 
to  find  faulty  SRUs  within  LRUs  even  when  the  SRUs  suffer  from 
Type  B  faults. 

Direct  Entry  into  Test  Sequences.  Most  tests  of  complex  avion¬ 
ics  at  the  shop  and  depot  involve  a  lengthy  sequence  that  always  runs 
in  the  same  order  and  often  requires  long  periods  of  time,  even  hours, 
to  complete.  Technicians  generally  cannot  directly  enter  the  test 
sequence  at  a  particular  point.  This  inhibits  their  ability  to  find  faults 
that  are  sensitive  to  time-varying  thermal  conditions,  especially  those 
faults  that  can  be  detected  only  during  the  early  stages  of  testing  when 
the  equipment  has  not  yet  reached  thermal  equilibrium. 

Contractor  engineers  were  able  to  devise  ways  to  directly  enter  some 
test  sequences  during  the  radar  R&M  program.  In  many  cases  this 
proved  to  be  an  effective  maintenance  aid. 

Loop  Testing.  Although  most  tests  of  complex  avionics  at  the  shop 
and  depot  are  more  detailed  and  thorough  than  those  in  the  airplane, 
one  exception  is  tests  run  by  the  continuous  monitor  BIT.  Depending 
on  the  mode  of  radar  operation,  the  repetition  frequency  for  BIT  tests 
may  be  high  or  low.  When  high,  the  BIT  has  the  advantage  of 
repeated  loop  testing  that  can  find  faults  with  intermittent  symptoms. 
If  technicians  in  the  shop  and  depot  could  achieve  comparable  test 
repetitions,  they  could  better  identify  and  correct  Type  B  faults. 

During  the  radar  R&M  program,  the  radar  contractors  found  that 
their  engineers  could  use  their  special  knowledge  of  test  equipment  to 
apply  the  direct  entry  ^chnique  and  to  institute  loop  testing  to  find 
intermittent  faults  that  had  eluded  routine  procedures. 

Next  we  consider  Priority  3  changes,  which  aim  at  improving  the 
ability  of  maintenance  technicians  at  all  levels  (flightline,  shop,  and 
depot)  to  identify  deficiencies  in  the  design  of  airborne  equipment  and 
ground-based  support  :quipment,  in  documentation  for  maintenance, 
and  in  the  debriefing  of  pilots. 

Material  Deficiency  Reporting.  The  Air  Force  has  a  formal  pro¬ 
cess  whereby  maintenance  technicians  can  report  deficiencies  in  the 
design  of  airborne  equipment  and  ground-based  support  equipment.21 
Each  radar  contractor  found  that  maintenance  technicians  often  do  not 
use  the  process.  Technicians  commonly  complained  that 

•  A  great  deal  of  time  usually  elapses  between  submitting  a  report 
and  receiving  a  response 

•  The  quality  of  the  response  is  usually  quite  low. 


21In  this  context,  equipment  includes  not  only  the  hardware  but  also  the  associated 
software. 
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Moreover,  Air  Force  personnel  responsible  for  responding  to  techni¬ 
cians’  reports  acknowledged  that  they  did  not  have  adequate  resources 
to  review  reports  and  initiate  corrective  actions.  The  entire  process 
needs  to  be  evaluated  and  appropriate  resources  applied  to  it. 

A  related  measure  that  could  help  the  process  is  tracking  of  individ¬ 
ual  items  of  equipment  following  maintenance.  (Each  item  has  a  dis¬ 
tinct  serial  number.)  Such  a  procedure  would  provide  information  that 
technicians  could  use  to  better  document  support  deficiencies. 

Technical  Orders  Feedback.  The  Air  Force  has  a  formal  process 
whereby  maintenance  technicians  can  report  deficiencies  in  the  TOs 
that  prescribe  how  maintenance  should  be  done.  The  radar  contractors 
reported  that  this  process  suffers  from  the  same  afflictions  as  the 
material  deficiency  reporting  process.  This  is  an  especially  serious 
situation  because  TOs  are  inevitably  immature  when  initially  fielded 
and  consequently  need  extensive  field  participation  for  their  improve¬ 
ment. 

During  the  radar  R&M  program,  maintenance  technicians  were  sel¬ 
dom  observed  referring  to  technical  manuals  except  when  dealing  with 
the  most  difficult  faults,  such  as  wiring  problems.  Again,  the  entire 
process  needs  to  be  evaluated  and  appropriate  resources  applied  to  it. 

Interactive  Pilot  Debriefing.  Both  radar  contractors  found  that 
debriefing  of  pilots  is  commonly  handled  by  maintenance  personnel 
who  lack  specialized  knowledge  about  the  radar  and  who  often  must 
debrief  multiple  sorties  by  several  pilots  at  a  given  time.  Because  of 
the  shortcomings  of  these  debriefings,  pilots  fail  to  provide  information 
necessary  to  identify  faults  and  fault  codes,  and  maintenance  personnel 
often  ignore  the  radar  malfunction  codes  produced  at  debriefing. 

In  particular,  debriefers  unfamiliar  with  the  radar  experience  great 
problems  using  the  Fault  Code  Manual.  During  the  radar  R&M  pro¬ 
gram,  debriefing  personnel  correctly  assigned  the  proper  far  It  code  only 
35  percent  of  the  time. 

Because  of  these  problems,  we  recommend  that  the  Air  Force 
explore  the  possibility  of  a  computer-aided  automated  system  that 
would  provide  a  hierarchical  menu  of  questions  concerning  malfunc¬ 
tions  reported  by  pilots.  Such  an  interactive  system  may  provide 
maintenance  technicians  better  information  for  trouble-shooting  the 
radar  system. 

Stage  2:  Implementation  of  Improvements 

At  p-esent  we  can  report  the  current  status  of  three  recommended 
R&M  improvements  common  to  both  radars: 
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•  A  prototype  of  a  system  for  tracking  equipment  by  serial  number 
(Priority  1)  is  undergoing  development  and  testing  at  Bitburg  Air 
Base  (see  Sec.  VI  for  further  details). 

•  As  an  interim  measure,  the  Air  Force  has  reduced  the  scope  of 
the  Material  Deficiency  Reporting  process  (Priority  3)  to  reduce 
the  quantity  of  MDRs  that  must  be  handled. 

•  A  prototype  of  an  interactive  pilot  feedback  debrief  (Priority  3)  is 
undergoing  evaluation  at  the  Air  Force  Logistics  Management 
Center. 


LESSONS  LEARNED 

From  these  exploratory  applications  we  learned: 

•  The  value  of  the  concept  of  maturational  development. 

•  Effective  procedures  for  applying  the  concept. 

Value  of  the  Concept 

Because  Stage  2  is  still  in  progress,  most  lessons  about  the  potential 
value  of  the  concept  must  be  inferred  from  the  Stage  1  results.  What 
was  learned  during  Stage  1?  What  benefits  and  costs  would  be 
expected  from  implementing  the  Stage  1  recommendations? 

Although  data  pointing  to  problems  with  the  F-16  LPRF  unit  and 
the  F-15  BIT  existed  long  before  the  1984  data  collection,  the  pre-1984 
data  lacked  depth,  context,  and  credibility.  Data  were  shallow  because 
the  prevalence  and  effect  of  the  problems  were  only  partially  docu¬ 
mented  and  the  relative  contributions  of  the  underlying  causes  were 
unknown.  The  context  of  data  was  incomplete  and  ambiguous  because 
not  all  R&M  deficiencies  and  their  interrelations  were  fully  assessed; 
consequently,  other  problems  appeared  far  more  dominant.  Data 
lacked  credibility  because  the  radar  development  contractors’  engineers 
were  mostly  detached  from  the  various  pre-1984  assessments. 
Although  no  new  major  problem  areas  were  discovered,  root  causes  of 
problems  were  identified  and  sorted  out  in  terms  of  their  effect.  The 
result  of  this  process  is  a  prioritized  package  of  improvements. 

Table  4.5  shows  estimates  for  the  main  costs  and  benefits  of  full 
incorporation  of  the  Priority  1  and  Priority  2  improvements  listed  in 
Tables  4.1  and  4.2.  The  main  benefit  lies  in  more  rapid  restoration  of 
combat-essential  equipment  to  a  state  of  full  designed  capability. 
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Table  4.5 

BENEFITS  AND  COSTS  OF  IMPLEMENTING  RECOMMENDED 
PRIORITY  1  AND  PRIORITY  2  IMPROVEMENTS 


Radar 

Mean  Time 
Between 
Confirmed 
Failure3 
(hours) 

Fault 

Removal 

Efficiency15 

(%) 

Additional 
Cost 
(million 
1984  $) 

APG  63  or.  the  F-15  C/D 
Observed  during  1984 

19 

21 

Projected  with 
improvements 

25 

50 

200 

APG  66  on  the  F-16  A/B 
Observed  during  1984 

82 

7 

Projected  with 
improvements 

100 

55 

250 

SOURCE:  Compiled  from  data  packages  that  Hughes  and  West- 
inghouse  provided  to  the  Strike  SPO. 

aMTBF  is  based  on  shop  confirmation  of  failure. 
bA  fault  removal  efficiency  of  21  percent  means  that  21  percent  of 
the  flights  with  a  radar  fault  indicated  ended  up  with  the  support 
process  removing  one  u-  more  faults. 


Effective  Procedures  for  Applying  the  Concept 

Given  that  a  subsystem  is  selected  for  maturational  development, 
what  lessons  can  we  draw  from  the  two  radar  applications?  What 
aspects  of  the  e  .oratory  application  were  especially  effective?  What 
aspects  might  be  strengthened  in  future  applications? 

Especially  Effective  Procedures.  The  Air  Force’s  execution  of 
the  exploratory  applications  of  the  concept  proved  to  be  especially 
effective  in  seven  respects: 

•  Stage  1  funding  with  resources  that  were  dedicated  to  assessing 
the  radar  R&M  situation  and  were  kept  independent  from  ongo¬ 
ing  radar  development  or  support  efforts. 

•  Stage  1  management  by  an  independent  SPO  (the  Strike  SPO). 

•  Strike  SPO  utilization  of  an  integrating  contractor22  to  help  over¬ 
see  and  evaluate  the  equipment  contractor  efforts. 


22Support  Systems  Associates,  Inc. 
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•  Engineering  assistance  supplied  to  the  Strike  SPO  by  the  Air 
Force  Acquisition  Logistics  Center. 

•  Direct  contracting  with  the  subsystem  contractors. 

•  Subsystem  subcontractors  contracting  with  the  weapon  system 
prime  contractors  for  engineering  support. 

•  Contractors  collecting  and  analyzing  data  on  their  own  equip¬ 
ment. 

Opportunities  to  strengthen  future  applications.  Two  aspects 
of  the  exploratory  applications  leave  opportunities  for  improvement  by 
future  applications  of  the  concept: 

•  Assure  an  earlier  start  for  Stage  1  by  making  a  commitment  to 
Stage  1  well  before  the  IOC  date  both  for 

—  Funds, 

—  Program  management  direction. 

•  Strive  for  more  timely  and  more  effective  implementation  of  the 
Stage  2  improvements  by 

—  Assigning  lead  responsibility  (and  staff)  to  a  single  organi¬ 
zation  (as  was  done  for  Stage  1), 

—  Reserving  funds  to  initiate  improvements. 

Even  with  such  strengthening  of  future  applications,  it  is  hard  to 
foresee  the  full  future  value  of  the  concept.  Very  clearly,  many  prob¬ 
lems  are  slipping  through  the  cracks  with  the  current  acquisition  pro¬ 
cess.  The  concept  of  a  formal  phase  for  maturational  development  is 
one  way  to  systematically  add 'ess  such  problems.  Applying  the  con¬ 
cept,  of  course,  is  not  trivial.  The  cost  for  each  application  was  $6  mil¬ 
lion  for  Stage  1,  and  each  application  would  run  from  $200  to  $250  mil¬ 
lion  for  full  implementation  of  Stage  2.  The  next  two  sections  address 
ways  to  reduce  such  costs  in  the  future. 


V.  EXISTING  INITIATIVES  THAT  CAN  HELP 
REFORM  AVIONICS  ACQUISITION 
AND  SUPPORT 


One  way  to  reduce  the  cost  of  maturational  development  is  to  reduce 
the  number  of  problems  that  slip  through  the  cracks  in  the  develop¬ 
ment  process.  The  Air  Force  already  has  several  initiatives  to  reduce 
R&M  deficiencies  before  the  IOC  date.  Strengthening  these  initiatives 
is  an  integral  part  of  our  proposed  strategy  for  reforming  avionics 
acquisition  and  support.  We  see  them  as  important  complements  to 
the  maturational  development  concept.  This  section  describes  these 
initiatives,  how  they  are  intended  to  improve  avionics  R&M,  and  how 
they  might  be  strengthened  to  increase  their  contributions. 

The  initiatives  fall  into  two  categories: 

•  Avionics  technologies. 

•  Development  procedures. 


R&M-RELATED  AVIONICS  TECHNOLOGIES 

The  margin  of  combat  superiority  that  our  weapon  systems  enjoy 
over  those  of  prospective  enemies  has  largely  resulted  from  our  aggres¬ 
sive  application  of  advanced  electronics  technologies  to  wartime  needs. 
Recognizing  our  strength  in  this  area,  the  Department  of  Defense  has 
embarked  on  programs  to  accelerate  this  trend  by  introducing  two  new 
basic  technologies  that  also  have  the  potential  to  contribute  to 
improved  R&M: 

•  Very  High  Speed  Integrated  Circuits  (VHSIC). 

•  Gallium  arsenide  circuits. 

Two  other  technological  developments  are  also  underway: 

•  Electronically  scanned  antenna. 

•  Improved  Built-in  Tests. 

The  electronically  scanned  antenna  may  appreciably  improve  the  reli¬ 
ability  of  fighter  airplane  radars  and  may  lead  to  the  replacement  of 
the  reliability-limited  mechanically  scanned  antenna.  Because  many 
other  avionics  technologies  will  contribute  to  the  R&M  states  for  the 
next  generation  of  fighter  airplanes,  we  recommend  that  the  Air  Force 
review  and  accelerate  other  R&M-related  avionics  technologies. 
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The  following  discussion  identifies  opportunities  that  the  Air  Force 
can  exploit  to  further  strengthen  the  H&M  benefit  to  be  derived  from 
these  new  technologies. 

Very  High  Speed  Integrated  Circuits  (VHSIC) 

The  use  of  VHSIC  aims  to  increase  digital  processing  capacity  through 
further  miniaturization  of  electronics,  which  will  increase  computing 
speed  by  shortening  the  paths  that  signals  must  traverse  and  increase 
computing  capacity  by  packing  more  equipment  in  a  given  amount  of 
space.  The  faster  computations  and  greater  number  of  computations  per 
unit  of  volume  promise  not  only  greatly  increased  functional  performance 
but  also  improved  ease  of  maintenance  and  reliability. 

Ease  of  Maintenance.  VHSIC  especially  offers  increased  oppor¬ 
tunities  to  track  down  Type  B  faults  by  capturing  information  about 
the  equipment’s  environment,  operating  state,  and  test  parameter 
values  at  the  time  of  the  failure.  However,  we  find  no  evidence  that 
VHSIC  technology  is  being  aggressively  applied  to  facilitate  collection 
and  analysis  of  such  information  to  support  the  detection  or  isolation 
of  subsystem-level  faults.  Thus  we  recommend  that  the  Air  Force  take 
measures  to  strengthen  the  application  of  VHSTC  to  improve 
subsystem-level  BIT. 

Reliability.  VHSIC  technology  also  offers  opportunities  to  improve 
reliability  because  design  and  manufacturing  goals  for  VHSIC  chips  are 
five  times  higher  than  those  for  existing  integrated  circuits,  and  placing 
more  circuits  on  a  single  integrated  circuit  chip  reduces  the  opportuni¬ 
ties  for  failure  at  points  where  chips  are  connected  to  adjoining  circuit 
elements.  However,  the  miniaturization  that  gives  VHSIC  its  vast 
computational  prowess  may  also  introduce  risks  that  could  cut  short 
such  projections.  Miniaturization  is  reaching  a  point  where  some 
electrical  elements  in  a  chip  are  only  20  to  100  atoms  thick.  We 
recommend  that  the  Air  Force  strengthen  efforts  that  are  exploring  the 
ability  of  such  chips  to  function  over  the  15-  to  25-year  service  life  of  a 
combat  airplane.1 

VHSIC  technology  does  not  necessarily  produce  subsystem  reliabil¬ 
ity.  VHSIC  technology  will  be  applied  mostly  in  those  areas  where 
digital  processing  is  now  performed.  However,  not  all  subsystems  are 
dominated  by  digital  processing,2  and  many  of  the  ones  that  are  will 

'The  Rome  Air  Development  Center  ia  currently  sponsoring  some  research  in  this 

area. 

2Thoae  subsystems  that  pose  the  greatest  R&M  problems  are  not  necessarily  dom¬ 
inated  by  failure  modes  in  the  digital  processing.  For  example,  less  than  30  percent  of 
the  removals  for  the  F-15  and  F-16  radars  involve  the  digital  processing  LRUs.  The  oth- 
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experience  great  increases  in  it  in  the  near  future.  Thus,  the  influence 
of  VHSIC  on  subsystem  reliability  will  depend  on  (1)  the  proportion  of 
the  subsystem  type  that  already  uses  digital  processing,  (2)  the 
increased  volume  of  such  processing  that  will  be  required  in  the  next 
generation  of  equipment,  and  (3)  the  dominant  failure  modes  that 
materialize  when  VHSIC  technology  meets  the  operational  environ¬ 
ment  of  the  combat  airplane. 

Although  these  potential  difficulties  should  not  raise  inappropriate 
alarm,  experience  with  innovation  has  taught  us  that  technological 
progress  in  electronics  can  introduce  new  failure  mechanisms  that  need 
to  be  identified,  diagnosed,  and  addressed  before  the  equipment  is 
mature.  Such  was  the  case  when  integrated  circuits  first  replaced  cir¬ 
cuits  constructed  from  transistors,  diodes,  etc.  At  that  time,  arguments 
for  integrated  circuits  were  similar  to  those  for  VHSIC:  fewer  elements 
and  connections  would  improve  reliability.  Initially,  however, 
integrated  circuits  led  to  lowered  reliability  because  manufacturers  did 
not  recognize  the  need  for  a  very  clean  manufacturing  environment.  In 
early  integrated  circuits,  minute  impurities  caused  molecular  migration 
that  corrupted  the  electrical  integrity  of  the  equipment.  In  addition, 
molecular  migration  can  afflict  much  larger  circuits,  such  as  those  in 
computers  and  in  the  F-16  radar’s  LPRF.  In  view  of  the  speed  with 
which  the  Air  Force  will  be  incorporating  VHSIC  technology  in  critical 
weapon  systems,  a  strengthening  of  its  efforts  to  identify  and  diagnose 
VHSIC  failure  modes  represents  an  opportunity  to  assure  that  VHSIC 
contributes  a  considerable  net  benefit  to  R&M.3 

Gallium  Arsenide  Circuits 

Gallium  arsenide  circuits  are  intended  to  increase  analog  processing 
capacity  the  same  way  that  VHSIC  aims  to  increase  digital  processing 
capacity.  Gallium  arsenide  technology  has  not,  however,  developed  as 
far  as  VHSIC  technology.  We  recommend  that  the  Air  Force  espe¬ 
cially  pursue  the  identification  and  diagnosis  of  failure  modes  that  may 
prove  peculiar  to  gallium  arsenide  circuits. 


era  are  in  the  analog  and  electromechanical  portions  of  the  radar.  Thus  there  are  limits 
to  the  R&M  benefits  derived  from  VHSIC, 

3Time  and  again,  development  programs  have  fallen  victim  to  what  might  be  termed 
the  “Titanic  Syndrome.”  Expected  to  be  invincible,  equipment  using  new  sophisticated 
technologies  has  unfortunately  fallen  prey  to  unforeseen  faults.  The  Air  Force  can  avoid 
catastrophic  R&M  “icebergs”  only  by  improving  our  knowledge  in  advance  about  failure 
pathologies  for  critical  new  technologies.  We  define  “failure  pathologies"  as  the  study  of 
the  essential  nature  of  the  conditions  under  which  technologies  tend  to  fail  and  the 
causes,  timing,  and  consequences  of  these  failures. 
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Reliability  will  improve  if  circuits  can  be  kept  at  low  temperatures. 
However,  the  need  for  lower  tempera;  ares  places  a  larger  demand  on 
the  airplane’s  environmental  cooling  system,  which  in  turn  increases 
weight  and  size.  Consequently,  we  recommend  that  the  Air  Force  care¬ 
fully  investigate  tradeoffs  between  reliability  improvements  due  to  gal¬ 
lium  arsenide  and  decreased  performance  due  to  greater  weight  and 
size.  In  addition,  we  recommend  an  aggressive  test  program  to  estab¬ 
lish  the  correlation  between  incremental  decreases  in  temperature  and 
incremental  increases  in  reliability.  For  R&M  to  fully  benefit  from  the 
application  of  gallium  arsenide  technology,  the  Air  Force  needs  to 
strengthen  ongoing  research  to  provide  a  sound  technical  base  from 
which  future  designers  can  make  appropriate  tradeoffs. 

Electronically  Scanned  Antenna 

An  electronically  scanned  antenna  for  a  fighter  airplane  would  have 
an  array  of  up  to  2000  electronically  active  elements.  Each  active 
phased  array  element  provides  the  equivalent  of  a  radar  antenna, 
transmitter,  and  receiver.  The  collection  of  elements  provides  an  elec¬ 
tronically  steerable  antenna  that  replaces  a  mechanically  scanned 
antenna.  An  active  phased  array  will  do  away  with  elements  in  current 
radars  that  are  prone  to  failure:  (1)  the  transmitter  with  its  high 
power  elements,  and  (2)  the  mechanical  and  hydraulic  elements  consti¬ 
tuting  the  antenna.  It  offers  two  benefits: 

•  Improved  radar  performance  because  of  the  more  rapid  scanning 
it  allows. 

•  Improved  R&M  because  it  replaces  equipment  that  currently  lim¬ 
its  radar  R&M.4 

Such  an  antenna  could  provide  considerable  benefits.  However,  con  • 
siderable  development  challenges  also  remain  in  this  area.  Because  of 
the  currently  limited  scope  and  amount  of  research  and  development 
efforts  sponsored  by  the  Air  Force  in  this  area,  we  recommend  an 
acceleration  of  such  efforts.  By  strengthening  research  and  develop¬ 
ment  of  an  electronically  scanned  antenna,  the  Air  Force  may  be  able 
to  more  quickly  identify  the  most  important  R&M  challenges  with  this 
important  new  technology.  The  sooner  such  challenges  are  defined,  the 
sooner  engineers  can  develop  sound  technical  approaches  for  dealing 

4There  is  also  the  opportunity  for  graceful  degradation  of  performance  as  elements  in 
the  array  experience  various  types  of  failures  because  the  failure  of  a  few  random  phased 
array  elements  does  not  materially  affect  radar  performance.  Some  failure  modes  may- 
lead  to  more  rapid  degradation  in  performance  than  others.  An  extreme  case  would  be 
where  many  neighboring  elements  failed  together. 
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with  them,  and  the  fewer  the  problems  that  will  remain  for  cleaning  up 
during  a  maturational  development  phase. 

Improved  Built-in  Tests 

Another  area  needing  increased  Air  Force  attention  is  BIT  technol¬ 
ogy.  Unlike  pilots  and  maintenance  personnel,  the  avionics  industry 
and  the  Air  Force  generally  fail  to  see  the  full  limitations  of  BIT. 
They  also  fail  to  see  its  great  unfulfilled  potential.  Avionics  contrac¬ 
tors  know  little  about  the  strengths  and  weaknesses  of  competing  BIT 
mechanizations  and  even  of  displays  for  BIT  results.  Indeed,  our 
experiences  have  shown  that  key  people  at  Westinghouse  do  not  fully 
understand  Hughes’s  approach  to  BIT  and  that  people  at  Hughes  do 
not  fully  understand  Westinghouse ’s.  In  addition,  no  Air  Force  organi¬ 
zation  oversees  BIT  development.  Thus  the  Air  Force  knows  neither 
the  current  state  of  the  art  nor  the  basic  research  needed  to  advance  it. 

The  Air  Force  especially  needs  to  accelerate  BIT  technology  to  deal 
more  rapidly  and  accurately  with  Type  B  faults.  As  a  consequence,  we 
propose  a  special  project— with  priority  for  expedited  research— to 
assess  the  state  of  the  art,  sponsor  research,  and  establish  design  guide¬ 
lines,  Design  guidelines  could  include  design  allocations  for  BIT,  sen¬ 
sors,  circuits,  and  computer  memory,  and  processing.  Most  important, 
BIT  mechanizations  need  to  capture  more  fully  the  state  of  the  subsys¬ 
tem  when  a  fault  occurs,  thus  capturing  key  system  parameters,  switch 
settings,  and  operating  environment  conditions.  Strengthening  the  Air 
Force’s  initiatives  directed  toward  BIT  technology  is  a  major  opportun¬ 
ity  for  improving  the  capability  of  the  support  process  to  address  Type 
B  faults. 

Other  Avionics  Technologies 

To  better  prepare  other  avionics  technologies  targeted  for  the  Air 
Force’s  next  generation  of  fighter  airplanes,  it  is  instructive  to  examine 
the  approach  being  taken  with  propulsion  technologies.  The  Engine 
SPO  is  currently  managing  or  coordinating  the  execution  of  programs 
aimed  at  cultivating  critical  technologies,  testing  them  in  exploratory 
development  tests,  and  maturing  them  in  advanced  development  pro¬ 
grams.  Similar  work  is  needed  for  avionics.  In  the  past,  advanced  tur¬ 
bine  engine  gas  generators  were  built  and  tested  to  pave  the  way  for 
subsequent  full-scale  engine  development.  Such  components  are 
usually  built  to  represent  the  expected  approximate  size  required  for 
the  target  application  engine.  These  advanced  designs  then  go  through 
further  changes  once  the  final  engine  size  has  been  determined. 
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Because  it  is  not  benefiting  from  such  a  degree  of  organized  atten¬ 
tion,  avionics  technology — also  crucial  to  the  next  fighter — faces  two 
serious  challenges: 

•  The  Air  Force  needs  to  review  the  status  of  critical  technologies, 
especially  radar  and  ECM,  and  identify  ways  of  accelerating  these 
technologies  to  benefit  R&M. 

•  The  Air  Force  needs  to  sponsor  more  efforts  devoted  to  building 
advanced  development  components  using  the  basic  technologies 
that  the  Avionics  Laboratory  and  avionics  industry  have  been 
cultivating. 

Such  components  can  then  receive  the  kind  of  performance  and  dura¬ 
bility  testing  that  critical  engine  components  now  receive.  By  so  doing, 
the  Air  Force  can  accelerate  the  maturation  of  technologies  and  equip¬ 
ment,  which  will  help  reduce  the  number  of  problems  that  would  arise 
during  a  maturational  development  phase. 


R&M-RELATED  DEVELOPMENT  PROCEDURES 

In  addition  to  the  foregoing  technology  initiatives,  the  Air  Force  is 
also  examining  new  procedures  aimed  exclusively  at  improving  the  ease 
of  maintenance  and  reliability  for  avionics  equipment.  This  subsection 
identifies  ways  that  three  of  these  procedures  could  be  strengthened  to 
make  an  even  larger  contribution  to  improved  R&M.  The  procedures 
address: 

•  Modular  avionics. 

•  Ultra-reliable  avionics. 

•  Avionics  integrity. 

Modular  Avionics 

Modular  avionics  aims  at  developing  smaller  and  cheaper  LRUs, 
which  in  turn  would  negate  the  need  for  an  avionics  shop.  The  whole 
intermediate  level  of  maintenance  might  be  eliminated. 

Current  LRUs  are  so  costly,  removed  so  often,  and  in  such  short 
supply  that  each  airbase  generally  needs  its  own  avionics  shop. 
Because  many  LRUs  cost  between  $100,000  and  $1,000,000,  the  avion¬ 
ics  shop  uses  large  sets  of  test  equipment  to  identify  faulty  SRUs 
within  these  LRUs.  The  shops  then  send  these  less  expensive  SRUs  to 
the  depot  for  repair,  which  reduces  the  time  and  the  value  of  assets 
tied  up  in  the  support  pipeline. 


75 


To  eliminate  the  need  for  avionics  shops  on  airbases,  the  avionics 
industry  and  various  Air  Force  organizations  are  examining  the  con¬ 
cept  of  modular  avionics.5  Rather  than  building  a  radar  with  five  to 
nine  LRUs  (see  Figs.  4.1  and  4.8),  these  efforts  aim  at  building  a  radar 
with  50  to  100 — or  perhaps  more — modules.  Like  LRUs,  these  modules 
could  be  removed  at  the  flightline;  like  SRUs,  these  modules  would  be 
fairly  cheap  and  could  be  sent  to  the  depot  for  repair. 

The  successful  development  of  modular  avionics  faces  two  major 
technical  hurdles: 

•  Packaging  the  modules  to  withstand  the  flight  and  flightline 
environment. 

•  Designing  BITs  to  isolate  faulty  modules  accurately. 

Packaging  Modules.  Flightline  maintenance  personnel  must  be 
able  to  remove  faulty  electronics  equipment  quickly  from  airplanes  not 
only  in  shelters  behind  closed  doors  but  also  outdoors  and  exposed  to 
wind,  rain,  snow,  dust,  and  even  sand.  Most  electronic  circuits  need 
protection  from  such  environmental  elements.  Current  LRUs  are  pro¬ 
tected  by  closed  metal  containers  that  are  only  opened  inside  the 
environmentally  controlled  avionics  shop.  For  modules  to  work,  they 
also  need  containers  to  protect  them  from  the  environment,  i.e.,  from 
wind,  rain,  snow,  dust,  and  sand,  and  this  will  inevitably  add  to  the 
airplane’s  weight,  volume,  and  environmental  cooling  needs.  In  addi¬ 
tion,  these  packaged  modules  need  to  be  plugged  into  the  airplane. 
Certain  types  of  connections,  such  as  those  for  the  F-16  radar’s  LPRF 
unit,  are  especially  demanding  in  terms  of  alignment,  firmness  of  fit, 
and  freedom  from  contamination.  Increases  in  the  number  of  such 
connections  and  in  the  environmental  stresses  placed  on  such  connec¬ 
tions  will  potentially  make  the  avionics  equipment  more  vulnerable  to 
the  flightline  environment. 

Designing  BIT.  Although  this  concept  of  sending  smaller  modules 
to  the  depot  is  economically  very  attractive,  even  avionics  shops  with 
their  large  sets  of  extremely  sophisticated  equipment  currently  experi¬ 
ence  problems  in  identifying  the  correct  SRU  to  send  to  the  depot. 
About  one  out  of  every  five  SRUs  sent  to  the  depot  has  no  failure  evi¬ 
dent  when  tested  on  the  depot’s  test  equipment.6  To  avoid  the  need  for 
an  avionics  shop  at  each  airbase,  adequate  amounts  of  BIT  need  to  be 
designed  to  isolate  the  faulty  modules  and  faulty  connections  between 

sThese  efforts  include  an  Air  Force  Avionics  Laboratory  program  known  as  PAVE 
PILLAR,  an  Air  Force  Air  Staff  effort  known  as  Modular  Avionics  System  Architecture, 
and  various  industry  efforts  known  as  Line  Replaceable  Modules. 

6Sending  the  wrong  module  to  the  depot  is  only  one  of  many  reasons  for  this  level  of 
performance. 
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modules.  Moreover,  some  way  must  be  found  to  deal  with  faults  that 
are  not  detected  and  isolated  by  the  BIT  down  to  the  module  level. 

Some  types  of  electronics,  such  as  digital  processors,  may  be  more 
amenable  to  modular  avionics.  Other  types,  such  as  those  with  delicate 
radio  frequency  connections  and  large  amounts  of  analog  circuitry,  may 
be  less  amenable  because  their  faults  are  so  difficult  to  isolate. 

To  enhance  its  decisionmaking  about  modular  avionics,  we  recom¬ 
mend  that  the  Air  Force  initiate  special  efforts  to  better  understand: 

•  Increases  in  an  airplane’s  weight,  volume,  and  environmental 
cooling  system  (ECS)  caused  by  modular  avionics. 

•  Requirements  for  connecting  modules  under  a  broad  range  of 
operating  environments. 

•  Ways  in  which  technology  can  provide  necessary  levels  of  fault 
isolation,  especially  in  modules  that  use  analog  circuits  or  that 
can  suffer  from  Type  B  faults. 

Although  research  is  lacking  or  is  proceeding  slowly,  modular  avion¬ 
ics  have  the  potential  to  affect  avionics  R&M  for  better  or  for  worse. 
It  is  important  therefore  that  a  sound  technical  base  be  developed  to 
support  future  decisions  in  this  area.  To  that  end,  we  see  a  need  for 
the  Air  Force  to  strengthen  its  research  and  development  in  this  area, 
especially  from  the  standpoint  of  maintainability. 

Ultra-Reliable  Avionics 

The  development  of  ultra-reliable  avionics  equipment  aims  at  pro¬ 
ducing  an  MTBF.of  from  2,000  to  10,000  hours  for  major  subsystems. 
This  advance  would  greatly  reduce  the  need  not  only  for  fault  isolation 
but  also  for  maintenance  at  all  levels  of  the  support  process.  To 
achieve  ultra-reliable  avionics,  the  Air  Force  is  looking  to  VHSIC  and 
gallium  arsenide  technologies  and  modular  avionics. 

Expectations  of  achieving  ultra-reliable  avionics  create  a  potentially 
dangerous  environment  for  improving  maintainability.  If  we  had  high 
assurance  that  such  subsystems  as  radars,  infrared  search  and  track 
sets,  ECM,  and  weapon  delivery  could  achieve  ultra-high  levels  of  reli¬ 
ability,  then  we  would  be  more  sympathetic  to  the  resulting  reduced 
emphasis  on  maintainability.  However,  our  previous  discussions  of 
VHSIC,  gallium  arsenide  circuits,  and  modular  avionics  have  raised 
enough  questions  to  cause  concern  about  placing  too  much  hope  in 
ultra-reliable  avionics  without  adequate  attention  to  fault  isolation. 

The  Air  Force  needs  to  strengthen  these  efforts  by  also  requiring  a 
balanced  emphasis  on  ultra-maintainability. 
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Avionics  Integrity  Program 

The  Aeronautical  Systems  Division  has  been  working  on  a  plan  for  a 
program  that  would  apply  to  avionics  the  principles  underlying  its 
Structural  and  Propulsion  Integrity  Programs.  These  principles  are 
embodied  in  a  three-step  process  that  involves: 

1.  Measurement  of  the  time-varying  stresses  and  temperatures  in 
the  operating  environment. 

2.  Determination  of  the  maximum  size  of  imperfections  that  may 
reasonably  be  expected  to  escape  detection  during  manufacturing 
and  be  tolerated  without  degrading  functional  performance. 

3.  Estimation  (with  models  of  physical  processes)  of  the  hours  of 
operational  service  that  would  have  to  occur  before  imperfec¬ 
tions  that  escaped  detection  during  manufacture  could  degrade 
needed  functional  performance. 

How  well  might  these  principles  be  applied  to  avionics? 

Engineers  need  better  information  about  the  time-varying  stresses 
and  temperatures  that  avionics  equipment  actually  experiences.  In  a 
combat  environment,  fighter  airplane  avionics  probably  operate  at 
higher  temperatures  and  under  wider  and  more  rapid  swings  in  tem¬ 
perature  and  g  loads  than  any  other  application  of  electronics.  These 
environmental  conditions  tend  to  shorten  the  life  of  electronic  com¬ 
ponents.  The  high  temperatures  result  from  the  large  amount  of 
avionics  equipment,  the  weight  and  volume  penalty  caused  by  increas¬ 
ing  the  size  of  the  ECS  that  cools  the  avionics,  and  the  added  drain  on 
valuable  propulsion  power  that  a  larger  ECS  would  impose.  The  high  g 
loads  in  a  fighter  can  rip  apart  a  connection  that  has  been  weakened 
through  cyclic  stresses  from  vibration  and  time-varying  g  loads  and 
thermal  gradients. 

The  wide  and  rapid  swings  in  thermal  and  g  loads  are  especially 
acute  in  combat-maneuvering  fighter  airplanes  that  dive  and  climb 
between  very  cold  high  altitudes  and  very  warm  low  altitudes.  In  many 
systems,  the  most  taxing  environmental  condition  is  a  rapid  dive  from 
a  high  altitude  followed  by  a  high  speed  dash  at  sea  level  on  a  hot  and 
humid  day.  The  airplane  is  being  buffeted  by  aerodynamic  forces,  and 
the  ECS  is  combating  not  only  internally  generated  heat  from  electron¬ 
ics  but  also  the  warm  ambient  condition  outside  the  airplane. 
Meanwhile,  it  is  “raining”  inside  the  avionics  bay  because  of  moisture 
condensation.  At  least  at  high  altitudes,  the  cold  outside  temperatures 
can  help  the  ECS  keep  the  electronics  somewhat  cooler  and  free  of 
moisture-induced  corrosion. 
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Designers  try  to  take  all  such  factors  into  account  when  designing 
electronics.  They  use  handbooks  and  generic  specifications  to  define  a 
baseline  characterization  of  the  operating  environment.  What  is  lack¬ 
ing  in  avionics  design— and  used  to  be  lacking  in  structures  and  propul¬ 
sion  design — is  a  follow-up  data  collection  effort  to  characterize  the 
actual  operating  environments.  This  follow-up  for  structures  and  pro¬ 
pulsion  has  shown  that  the  actual  operating  environments  were  far 
harsher  than  the  baseline  characterization  used  for  design;  further¬ 
more,  such  environments  were  shortening  the  functional  life  of  key 
components. 

Because  avionics  design  may  experience  similar  problems,  the  ASD 
plan  to  require  routine  monitoring  of  temperature  and  stress  could  be 
highly  beneficial  to  controlling  future  design  choices  regarding  heat  dis¬ 
sipation  features  and  required  ECS  capacities.  It  should  add  only 
small  marginal  costs. 

Although  the  first  basic  principle  underlying  the  Structural  and  Pro¬ 
pulsion  Integrity  programs  appears  to  have  directly  beneficial  relevance 
to  avionics,  it  is  less  clear  that  the  other  two  will  be  as  helpful,  espe¬ 
cially  for  leading-edge  technologies  where  designers  are  applying  recent 
technological  advances  for  the  first  time.  Some  degree  of  operational 
experience  is  often  required  before  the  operationally  dominant  failure 
modes  and  failure  physics  are  understood  well  enough  to  support  ana¬ 
lytic  modeling  and  engineering  evaluation  during  the  design  process. 
Because  electronics  equipment  uses  many  different  materials  and 
manufacturing  processes,  we  still  lack  much  information  about 
manufacturing  impurities,  their  critical  sizes,  and  the  rates  at  which 
they  enlarge  and  migrate  when  subjected  to  different  time-varying 
stresses. 

For  mechanical  failure  modes  of  well-known  technologies,  the  kinds 
of  combined  thermal  and  stress  analysis  advocated  by  the  Integrity 
programs  certainly  need  to  be  extended  to  avionics.  The  appropriate 
approach  for  other  failure  mechanisms — such  as  chemical  processes, 
connector  problems,  and  system  faults — is  less  clear. 

The  primary  means  that  the  Air  Force  has  to  strengthen  the  contri¬ 
butions  of  the  Avionics  Integrity  Program  (AVIP)  toward  improved 
R&M  are 

•  Accelerate  efforts  to  characterize  actual  operating  environments. 

•  Expand  AVIP  to  include  research  into  the  potential  linkage 
between  time-varying  environmental  conditions  and  the  time- 
varying  symptoms  of  some  Type  B  faults. 

Such  actions  would  help  advance  the  state  of  knowledge  regarding  the 
phenomenon  of  the  Type  B  fault  and  its  influence  on  maintainability. 
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Summary 

New  technologies  such  as  VHSIC,  electronically  scanned  antennas, 
and  gallium  arsenide,  and  new  procedures  such  as  modular  avionics, 
ultra- reliable  avionics,  and  the  Avionics  Integrity  Program  can  all  help 
improve  avionics  R&M.  Moreover,  by  strengthening  ongoing  work  in 
each  of  these  areas,  the  Air  Force  can  further  improve  avionics  R&M. 
We  see  such  strengthening  as  an  important  part  of  an  overall  strategy 
to  reform  avionics  acquisition  and  support. 

Even  with  strengthened  initiatives  we  must  keep  our  expectations  in 
perspective.  We  need  time  to  acquire  engineering  data,  develop  ana¬ 
lytic  methods,  and  institute  necessary  guidance.  Perhaps  quantum 
improvements  in  reliability  will  prove  attainable  and  affordable  even  in 
the  fighter  airplane  environment,  and  perhaps  someday  such  improve¬ 
ments  will  regularly  be  achieved.  Meanwhile,  we  continue  to  need 
improved  maintainability,  including  the  development  of  more  effective 
BIT.  Thus  far,  the  current  Air  Force  activities  described  in  this  sec¬ 
tion  have  not  addressed  ease  of  maintenance  with  the  same  depth  that 
they  have  addressed  reliability  (see  Table  5.1).  A  more  balanced 
approach  is  needed  that  emphasizes  both  reliability  and  maintainabil¬ 
ity.  One  way  to  help  provide  that  balance  is  the  maturational  develop¬ 
ment  concept  that  was  discussed  in  Secs.  Ill  and  IV. 

Table  5.1 

SUMMARY  OF  THE  RELATIVE  POTENTIAL  CONTRIBUTIONS  FROM 
CURRENT  AIR  FORCE  INITIATIVES 


Reliability  Maintainability 


Current  R&M-Related  Initiatives 

Type  A 
faults 

Type  B 
faults 

Type  A 
faults 

Type  B 
faults 

Technologies 

VHSIC 

+  +  + 

+ 

+ 

+ 

Gallium  arsenide  circuits 

+  +  + 

+ 

Electronically  scanned  antenna 

+  +  + 

+ 

+ 

BIT 

+ 

+ 

Other  fighter  avionics 

+  +  + 

+ 

Development  Procedures 

Modular  avionics8 

Ultra-reliable  avionics 

+  +  + 

+ 

Avionics  integrity  program 

+  +  + 

+ 

NOTE:  Plus  signs  (+)  denote  relative  degrees  of  expected  contribution. 
Three  (+++)  denote  a  greater  expected  contribution  than  two  (++). 

aThe  contribution  from  modular  avionics  lies  mainly  in  the  reduced  cost 
of  the  flightline-removable  module  and  the  opportunity  to  avoid  having  an 
avionics  shop  at  the  base.  The  real  problem  with  current  avionics  involves 
isolating  faults  to  the  proper  module. 


VI.  NEW  INITIATIVES  NEEDED  TO  HELP 
REFORM  AVIONICS  ACQUISITION 
AND  SUPPORT 


To  help  address  the  major  weaknesses  in  the  processes  for  avionics 
acquisition  and  support  (Sec.  II),  the  preceding  sections  have  proposed 
a  complementary  combination  of  instituting  maturational  development 
and  strengthening  certain  initiatives  that  the  Air  Force  has  already 
undertaken.  Two  further  initiatives  would  provide  the  Air  Force  with 
major  additional  st'-  toward  implementation  of  a  comprehensive 
strategy  for  reshaping  its  processes  for  avionics  acquisition  and  sup¬ 
port.  To  complete  this  strategy,  we  recommend  that  the  Air  Force: 

•  Institute  Performance  Oriented  Tracking  of  Equipment  Repair 
(PORTER). 

•  Reorganize  its  avionics  engineering  resources. 

These  initiatives  could  help  ensure  a  balanced  approach  to  avionics 
acquisition  and  support,  one  that  appropriately  concentrates  on  both 
reliability  and  maintainability.  They  also  could  go  a  long  way  toward 
making  maturational  development  a  more  cost-effective  process. 


INSTITUTE  PERFORMANCE  ORIENTED  TRACKING 
OF  EQUIPMENT  REPAIR  (PORTER) 

This  initiative  has  the  potential  to 

•  Greatly  increase  the  capability  of  maintenance  technicians  to 
identify  equipment  that  the  shop  or  depot  has  failed  to  restore  to 
full  designed  capability. 

•  Considerably  reduce  the  cost  of  collecting  data  for  Stage  1  of 
maturational  development. 

A  prototype  has  already  demonstrated  many  of  the  functions  essential 
to  a  full  mechanization  of  the  PORTER  concept.  The  Air  Force 
currently  is  exploring  the  possibility  of  instituting  the  PORTER  con¬ 
cept  within  its  recently  developed  Core  Automated  Maintenance  Sys¬ 
tem  (CAMS). 

This  subsection  defines  the  problem  that  PORTER  was  conceived  to 
address,  discusses  the  PORTER  concept,  describes  the  evolution  of  the 
prototype,  reviews  some  preliminary  results,  and  proposes  a  plan  for 
instituting  the  PORTER  concept. 
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Problem 

Even  with  the  best  engineering  efforts — including  a  formal  process 
aimed  at  maturing  reliability  and  maintainability — some  avionics  faults 
will  still  evade  detection  and  correction  at  various  points  in  the  support 
process.  The  experimental  prototype  system  known  as  PORTER  aims 
at  providing  information  needed  to  track  down  such  faults,  fix  them, 
and  in  so  doing  document  the  deficiencies  in  the  support  process  that 
allowed  these  faults  to  escape  detection  in  the  first  place. 

Faults  in  avionics  equipment  can  escape  detection  for  many  reasons. 
On  the  one  hand,  the  narrow  confines  of  the  airplane  and  the  limita¬ 
tions  of  BITs  make  it  impossible  to  test  all  equipment  thoroughly  when 
the  airplane  is  in  the  air.  On  the  other  hand,  the  inability  to  replicate 
airborne  stresses  makes  it  impossible  to  test  all  equipment  realistically 
when  the  airplane  is  on  the  ground.  Moreover,  when  equipment  is 
removed  from  the  airplane  and  taken  to  shops  and  depots  for  more 
exhaustive  testing,  both  the  nature  of  the  tests  and  their  pass/fail  cri¬ 
teria  change. 

Because  of  the  complexity  of  the  avionics  equipment,  tests  often 
inadvertently  leave  gaps  in  their  coverage.  Consequently,  faulty  equip¬ 
ment  circulates  through  the  support  system  until  either  the  fault 
develops  into  a  more  serious  problem  or  maintenance  personnel  take 
extraordinary  measures  to  isolate  the  problem.  Such  repeated  circula¬ 
tion  of  faulty  equipment  occurs  all  too  frequently  with  sophisticated 
airborne  electronics  equipment. 

When  an  item  of  equipment  circulates  in  this  fashion,  it  is  com¬ 
monly  labeled  a  “bad  actor.”  These  must  be  eliminated  because  they 
deny  pilots  regular  and  dependable  access  to  the  equipment’s  full 
designed  capability.  Fixing  them  involves  extra  effort  on  the  flightline, 
in  the  shop,  and  at  the  depot.  Bad  aciors  must  be  located  and  the 
LRUs  must  be  then  sent  to  the  depot  for  special  repair  action  so  that 
airplanes  are  as  completely  rid  of  them  as  possible  and  are  restored  to 
their  full  combat  capability.  Eliminating  bad  actors  will  reduce  the 
extraordinary  maintenance  effort  on  the  flightline,  in  the  shop,  and  at 
the  depot,  but  most  important,  it  will  provide  pilots  with  equipment 
that  can  dependably  deliver  its  full  designed  capabilities. 

Unfortunately,  the  Air  Force’s  standard  data  systems  do  not  gather 
sufficient  information  from  flight-to-flight  performance  to  enable 
maintenance  personnel  to  identify  and  diagnose  ineffectual  mainte¬ 
nance  and  evolving  problems  with  sophisticated  avionics.  PORTER 
attempts  to  fill  this  gap. 
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The  Concept  of  PORTER 

PORTER's  primary  purpose  is  to  be  a  maintenance  aid.  It  aims  at 
reducing  avionics  maintenance  burdens— especially  for  such  sophisti¬ 
cated  equipment  as  fire  control  radars — through  judicious  collection 
and  timely  transmission  of  performance  and  maintenance  information 
to  critical  points  in  the  support  process.  This  information  should 
enable  maintenance  personnel  to  identify  and  quickly  fix  equipment 
that  otherwise  circulates  repeatedly  through  the  support  process,  using 
up  time  and  scarce  resources.  Figure  6.1  shows  how  maintenance 
information  currently  flows,  and  Fig.  6.2  shows  how  it  would  flow  after 
PORTER  were  instituted. 

Ultimately,  the  use  of  PORTER  should  increase  the  number  of 
weapon  systems  prepared  to  deliver  their  full  designed  capability. 
Thus,  the  philosophy  underlying  PORTER  is  to  increase  combat  effec¬ 
tiveness  by  decreasing  the  burden  on  personnel,  spare  parts,  and  test 
equipment. 

For  PORTER  to  work,  the  perceived  value  of  its  services  must 
exceed  the  costs  imposed  by  its  record-keeping  demands.  Thus,  every 


Fig.  6.1 — Current  maintenance  information  flow 


NOTES:  OF  1,  FF.1.SF.1  and  SF.2  are  computer  files. 
PR  1  and  Ffl  1  are  printed  reports 


Fig.  6.2— Maintenance  information  flow  after  instituting  PORTER 


supplier  of  information  should  receive  a  directly  beneficial  product  that 
logically  depends  on  the  information  he  supplies  (see  Table  6.1).  For 
example,  although  the  pilot  would  spend  slightly  more  time  in  mainte¬ 
nance  debrief,  he  would  receive  a  preflight  advisory  that  will  make  him 
more  aware  of  potential  problems  with  his  airplane  and  enable  him  to 
make  more  effective  maintenance  requests. 

Although  PORTER  should  provide  uniquely  useful  maintenance 
information  for  product  improvement,  its  primary  goal  is  to  serve  as  a 
maintenance  tool,  not  as  a  data  system.  This  goal  places  serious  con¬ 
straints  on  PORTER’s  development.  It  must  be  highly  synchronized 
with  actual  events  in  the  maintenance  process.  To  accomplish  this,  an 
operational  squadron  has  been  closely  involved  not  only  in  testing  and 
debugging  the  prototype  for  PORTER,  but  also  in  defining  and  experi¬ 
menting  with  its  specifications. 
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Table  6.1 

PORTER’S  PRODUCTS  AND  BENEFITS 


Supplier  of 
Information 

Product 

Received 

Benefit 

Perceived 

Pilot 

Preflight  advisory 

Sit'  .'tion  awareness 

Postflight  fault 
history 

More  effective  maintenance 
requests 

Debriefer 

Fault  history  by 
subsystem 

Fewer  fault  entries  due  io 
more  timely  maintenance 
requests 

Flightline 

Maintenance 

Composite  history  of 
faults,  maintenance 
actions,  and 
resident  LRUs 

Fewer  LRU  removals  due 
to  more  rapid  fault 
isolation 

Shop 

Maintenance 

Composite  history  of 
LRUs 

Quicker  case  development 
for  bad  actor  LRUs 

Evolution  of  the  PORTER  Prototype 

When  the  results  of  the  F-15/F-16  Radar  R&M  Improvement  Pro¬ 
gram  were  briefed  during  October  1985  to  Headquarters  United  States 
Air  Forces  in  Europe  (USAFE)  and  the  36th  TFW  at  Bitburg,  audi¬ 
ences  were  receptive  to  the  recommendation  concerning  detailed 
debriefing  of  pilots  and  tracking  of  equipment  by  serial  number.  Con¬ 
sequently,  a  prototype  development  and  demonstration  of  PORTER 
was  instituted  at  Bitburg.  This  effort  had  two  main  objectives: 

•  Demonstrate  an  initial  PORTER  capability  at  Bitburg  Air  Base. 

•  Suggest  how  specifications  for  future  Air  Force  data  systems 
might  include  PORTER-like  services. 

Because  the  prototype  development  and  demonstration  effort  was 
conceived  as  a  short-term,  quick-response  project,  it  did  not  enjoy  the 
benefits  of  an  in-depth  research  project  that  could  have  explored  alter¬ 
native  architectures,  hardware,  and  software  approaches.  The  project 
had  the  advantage,  however,  of  a  joint  development  enterprise  that 
includes  technicians  in  the  field — the  people  PORTER  would  serve. 

The  TORTER  prototype  was  conceived  as  a  personal-computer- 
based  (PC)  tool  to  help  maintenance  technicians.  Whether  it  would 
help,  and  whether  it  would  be  used,  depends  on  how  maintenance 
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personnel  would  use  PCs  to  enter,  transmit,  and  extract  data.  Success 
of  the  PORTER  prototype,  therefore,  depends  on  finding  ways  for  PCs 
to: 

1.  Accommodate  the  breadth  of  subsystems,  the  depth  of  informa¬ 
tion,  and  the  amount  of  history  that  maintenance  technicians 
would  require. 

2.  Overcome  the  slowness  with  which  standard  database  packages 
extract  information  from  large  databases. 

3.  Store  large  amounts  of  information  economically  when  relevant 
data  forms  are  either  mostly  blank  or  (less  frequently)  contain 
large  amounts  of  useful  information. 

4.  Efficiently  transfer  information  between  maintenance  levels 
(cockpit,  debriefing,  flightline,  shop,  and  depot). 

USAFE  added  a  fifth  challenge  in  response  to  their  concern  about  a 
prospective  enemy’s  gaining  access  to  information  that  would  reveal 
weaknesses  in  combat  capability: 

5.  Provide  adequate  protection  (e.g.,  encryption)  for  sensitive  data. 

More  recently,  a  sixth  challenge  is  emerging  with  the  incorporation  of 
the  Core  Automated  Maintenance  System  (CAMS)  at  Bitburg  during 
the  summer  of  1987: 

6.  Interface  and  integrate  a  PORTER  capability  with  CAMS  to 
avoid  duplication  of  data  entry  tasks  and  equipment. 

Preliminary  Results 

The  prototype  has  thus  far  met  challenges  1,  2,  and  51  and  thereby 
provided  Bitburg  with  a  limited  initial  capability  and  the  Air  Force 
with  some  initial  ideas  about  potential  specifications  for  enhancing 
CAMS. 

With  this  initial  capability,  the  36th  TFW  at  Bitburg  has  developed 
a  database  that  is  being  used  to 

•  Identify  LRUs  with  histories  of  repeated  visits  to  the  shop. 

•  Develop  documentation  packages  that  are  being  sent  to  the  depot 
whenever  such  an  identified  LRU  is  sent  to  the  depot. 

Thus  far  three  such  LRUs  and  associated  documentation  packages 
have  been  sent  to  the  depot  at  the  Warner  Robins  ALC. 

'Challenge  5  was  satisfied  by  using  TEMPEST  qualified  PCs  and  by  using  floppy 
disks  to  manually  pass  data  between  locations  on  base. 
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Example  1.  The  first  of  the  three  LRUs  left  Bitburg’s  avionics 
shop  on  December  13,  1986.  During  May  1987,  it  finished  receiving 
depot  repairs  and  was  on  its  way  back  to  the  field.  It  is  a  unit  that 

•  Accumulated  52  operating  hours  (approximately  40  flights)  while 
installed  on  various  aircraft  during  1986. 

•  Would  intermittently  fail  either  the  “blanking  gate”  BIT  or  the 
“RF  No  Go”  BIT  while  installed  on  various  aircraft  during  1986. 

•  Made  nine  visits  to  the  base  avionics  shop  during  that  time, 
where  an  additional  75  operating  hours  were  accumulated. 

At  Bitburg,  a  Hughes  technical  representative  helped  the  shop  techni¬ 
cians  in  their  attempts  to  repair  the  LRU.  When  the  unit  arrived  at 
the  depot,  the  technicians  undertook  extraordinary  measures  to  try  and 
fix  the  unit: 

•  They  ran  the  unit  for  two  days  on  the  AIS  test  equipment2  before 
they  detected  and  isolated  the  first  failure  (“blanking  gate”)  to  an 
SRU.  Shortly  thereafter  they  detected  and  isolated  faults  con¬ 
sistent  with  the  RF  No  Go  failures.  These  faults  were  located  on 
two  other  SRUs. 

•  After  replacing  these  three  SRUs,  they  then  installed  the  unit  on 
the  subsystem  bench3  and  ran  150  iterations  of  the  portion  of  the 
BIT  that  tests  this  unit  to  further  verify  their  repairs. 

Whether  the  depot’s  very  considerable  efforts  have  finally  fixed  this 
unit  will  not  be  known  until  it  reenters  operational  service  at  some  air 
base.  In  an  attempt  to  follow  up  on  the  performance  of  the  LRU,  a 
note  on  a  tag  tied  to  the  unit  asks  the  base  that  receives  it  to  contact  a 
person  at  the  depot  so  that  technicians  can  track  the  unit’s  perfor¬ 
mance.  (If  all  F-15  bases  had  a  PORTER  capability,  such  tracking 
would  occur  almost  automatically.) 

Example  2.  This  unit  left  Bitburg’s  avionics  shop  on  April  6,  1987, 
and  arrived  at  the  depot’s  repair  line  on  May  29,  1987,  with  its  com¬ 
plete  complement  of  PORTER  documentation  showing  that  the  unit 

•  Accumulated  56  operating  hours  (approximately  40  flights)  while 
installed  on  various  aircraft  during  1986. 

2Using  the  same  type  of  test  equipment  and  the  same  tests  as  used  at  Bitburg’s  avion¬ 
ics  shop. 

3The  shop  at  Bitburg  does  not  have  a  subsystem  bench.  Such  a  bench  is  a  complete 
radar  subsystem,  with  some  associated  test  equipment,  that  provides  a  subsystem-level 
environment  for  testing.  The  shop’s  test  equipment  only  tests  LRUs  individually,  in  iso¬ 
lation  from  the  rest  of  the  radar  subsystem. 


87 


•  Made  six  visits  to  the  shop  during  that  time,  where  an  additional 
16  operating  hours  were  accumulated. 

The  depot  was  working  on  this  unit  at  the  time  of  publication  of  this 
report. 

Example  3.  This  unit  left  Bitburg’s  avionics  shop  on  January  22, 
1987.  The  Hughes  technical  representative  at  Bitburg  alerted  the 
depot  that  this  unit  was  on  the  way.  The  PORTER  documentation 
showed  that  the  unit 

•  Accumulated  18  operating  hours  (approximately  15  flights)  while 
installed  on  various  aircraft  during  1986. 

•  Made  seven  visits  to  the  shop  during  that  time,  where  an  addi¬ 
tional  eight  operating  hours  were  accumulated. 

Unfortunately,  the  unit  went  through  the  depot’s  standard  LRU  repair 
procedures  and  left  the  depot  before  anyone  realized  that  it  was  a 
PORTER  item.  The  depot’s  limited  documentation  does  not  reveal  the 
nature  of  the  depot’s  repair  action.  Moreover,  the  lack  of  a  worldwide 
PORTER  capability  means  that  we  cannot  follow  up  on  the  unit’s  per¬ 
formance  at  its  next  air  base. 

A  Plan  for  Instituting  PORTER 

A  general  implementation  of  PORTER  has  the  potential  of  helping 
maintenance  technicians  better  maintain  the  sophisticated  kinds  of 
electronics  that  are  now  an  integral  part  of  nearly  every  type  of  aircraft 
that  the  Air  Force  now  operates.  We  propose,  therefore,  that  the  Air 
Force  institute  PORTER  in  a  manner  that  gives  it  the  option  to  install 
a  PORTER  capability  initially  at  the  bases  in  greatest  need  and  then 
later  at  other  bases. 

As  PORTER  is  applied  to  different  weapon  systems,  the  software 
will  need  to  be  tailored  to  accommodate  differences  in 

•  The  complement  of  subsystems  that  constitute  the  weapon  sys¬ 
tems. 

•  The  pods  that  the  weapon  systems  carry. 

•  Failure  indicators  (including  BIT)  for  subsystems. 

•  Maintenance  parameters  peculiar  to  a  specific  subsystem. 

The  prototype  software  in  use  at  Bitburg  for  the  F-15  cannot  be  used, 
for  example,  on  the  F-16,  F-lll,  etc.  This  leads  to  challenge  number  7 
for  implementing  PORTER: 

7.  Tailor  the  software  for  an  individual  weapon  system. 
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The  Proposed  Architecture.  One  way  to  meet  the  seven  chal¬ 
lenges  is  to  use  PCs  to  simultaneously  provide 

•  PORTER  services. 

•  Data  entry  and  extraction  terminals  for  CAMS. 

The  PCs  could  be  placed  at  seven  locations,  as  they  are  now  with  the 
prototype  at  Bitburg:  pilot  debriefing  (three  locations,  one  for  each 
squadron),  flightline  maintenance  (three  locations,  one  for  each  squad¬ 
ron),  and  the  avionics  shop.4  The  PCs  would  need  to  communicate  with 
one  another  and  with  the  CAMS  software  that  is  now  hosted  on  a 
mainframe  computer  (the  Sperry  1100). 

An  Alternative  Architecture.  Another  way  to  meet  the  seven 
challenges  is  to  build  the  PORTER  services  into  the  CAMS  software 
that  would  be  hosted  at  a  centralized  location(s).  With  this  architec¬ 
ture,  user  locations  could  be  served  by  dumb  terminals.  There  would 
be  no  need  for  PCs.  There  would  be  no  need  for  maintaining  PC 
software.  There  would  be  no  need  for  communication  links  other  than 
the  direct  links  to  the  CAMS  centralized  location(s).  Although  these 
are  certainly  attractive  features,  the  prospective  disadvantages  are 

•  Time  required  for  software  development. 

•  Time  required  to  adapt  software  to  the  evolving  definition  of 
PORTER  services. 

•  Loss  of  air  base  flexibility  to  tailor  PORTER  services. 

•  Potential  nonavailability  of  the  CAMS  centralized  location(s) 
because  of  deployment  or  enemy  attack  on  the  air  base  or  other 
problems  at  the  centralized  location(s). 

Current  CAMS  Architecture.  Another  approach  is  to  incorporate 
as  much  of  the  PORTER  services  as  possible  within  the  current  CAMS 
architecture.  The  Air  Force  plans  a  test  of  this  approach  during  the 
summer  of  1987  at  Tyndall  Air  Force  Base.  This  approach  would  give 
the  Air  Force  a  nearly  immediate  capability.  The  prospective  disad¬ 
vantage  is  that  only  a  small  portion  of  the  PORTER  services  may  be 
provided.  The  summer  1987  test  needs  to  help  answer  three  questions: 
(1)  How  much  of  the  PORTER  concept  can  be  delivered  quickly  by 
merely  adding  an  analysis  package  and  retrievals  to  the  current  version 
of  CAMS?  (2)  How  far  does  such  a  quick  mechanization  go  toward 
giving  the  Air  Force  the  information  management  capability  that  it 
needs  to  effectively  identify  and  repair  bad  actor  equipment?  (3)  Is 
further  progress  needed? 

4For  the  avionics  shop,  it  probably  would  be  worthwhile  to  have  three  PCs:  one  for 
the  automatic  test  stations,  one  for  manuals,  and  one  for  the  TEWS  test  equipment. 
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Recommendation.  In  the  event  that  the  answer  to  question  three 
is  yes,  and  in  view  of  the  reality  that  the  test  may  take  longer  than 
expected  to  complete,  we  recommend  that  the  Air  Force  concurrently 
begin  preliminary  definition  for  the  Proposed  Architecture  and  also 
review  the  Alternative  Architecture  as  a  backup. 

There  are  two  justifications  for  such  an  accelerated  effort:  (1)  the 
value  in  combat  from  increased  readiness  of  full  designed  capabilities, 
and  (2)  the  opportunity  to  appreciably  reduce  the  cost  of  collecting  the 
data  needed  for  Stage  1  applications  of  maturational  development. 
Much  of  the  $6  million  cost  for  assessing  the  R&M  situation  for  each 
radar  (Sec.  IV)  was  accounted  for  by  the  17  data  collectors  at  each  air 
base.  If  those  bases  had  a  PORTER  capability  already  resident  and  in 
full  use,  the  number  of  data  collectors  could  have  been  considerably 
reduced.  Early  instituting  of  PORTER,  therefore,  can  make  Stage  1  of 
the  maturational  development  concept  more  affordable  sooner  and 
increase  the  overall  effectiveness  of  the  concept. 


REORGANIZE  AVIONICS  ENGINEERING  RESOURCES 

The  overall  effectiveness  of  maturational  development  is  also  influ¬ 
enced  to  a  very  great  degree  by  how  the  Air  Force  applies  its  limited 
avionics  engineering  resources.  From  the  previous  sections  one  can 
infer  several  difficulties  with  the  application  of  these  resources.  After 
reviewing  the  more  dominant  difficulties,  we  set  forth  a  proposal  for 
reorganizing  the  application  of  these  resources. 

Difficulties 

Diffusion  of  R&M  Responsibilities.  Diffusion  of  R&M  responsi¬ 
bilities  occurs  throughout  the  acquisition  process.  It  becomes  espe¬ 
cially  acute  when  the  Weapon  System  SPO  starts  anticipating  PMRT. 
By  its  very  nature,  the  advent  of  PMRT  forces  the  Weapon  System 
SPO  to  start  closing  out  its  engineering  responsibilities.  It  is  therefore 
the  wrong  time  to  start  new  R&M  improvement  initiatives.  Moreover, 
once  PMRT  occurs,  responsibility  for  an  avionics  subsystem  passes  to 
one  organization  and  responsibility  for  the  shop’s  intermediate  test 
equipment  passes  to  an  entirely  different  one,  usually  located  at  a  dif¬ 
ferent  air  base. 

Lack  of  a  Single  Organization  to  Manage  Implementation  of 
Improvements.  For  the  ongoing  efforts  to  mature  the  F-15  C/D  radar 
and  the  F-16  A/B  radar,  the  ASD  Strike  SPO  was  made  responsible  for 
the  Stage  1  assessment  of  the  R&M  situation,  but  no  single 
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organization  bears  responsibility  for  coordinating  Stage  2  (implementa¬ 
tion  of  improvements)  for  either  radar.  The  Strike  SPO  has  helped  fill 
the  gap  on  several  of  the  generic  improvements.  However,  resource 
limitations  of  the  Strike  SPO  together  with  personnel  limitations  at 
the  depots  have  constrained  the  extent  of  this  involvement  and  thus 
restricted  the  pace  of  the  implementation. 

Not  Allowing  Depots  to  Use  Development  Funds  to  Under¬ 
take  Engineering  Development  of  Improvements.  One  limitation 
at  the  depot  is  the  policy  of  not  allowing  depots  to  spend  development 
funds  (so-called  3600  money)  to  fund  engineering  development  of 
needed  improvements.  This  has  hindered  development  of  a  new  LPRF 
unit  for  the  F-16  A/B  radar.  Where  exceptions  to  this  policy  have 
been  allowed,  progress  is  now  being  made. 

A  Model  for  an  Avionics  Engineering  Center 

Avionics  engineering  resources  should  be  reorganized  to  address  the 
foregoing  difficulties  and  to  improve  the  Air  Force’s  ability  to  imple¬ 
ment  the  proposals  put  forth  in  this  report.  One  way  to  do  this  would 
be  to  create  an  Air  Force  Avionics  Engineering  Center  along  the  lines 
of  the  following  model. 

The  Center  would  provide  broadly  ranging  expertise  and  corporate 
memory,  including  detailed  knowledge  of  forthcoming  threats,  R&M 
problems  in  the  field,  ongoing  development  efforts,  and  potential  roles 
for  emerging  technologies.  It  would  apply  such  a  knowledge  base  and 
its  engineering  expertise  to  Air  Force  avionics  efforts  ranging  from 
advanced  basic  research  through  maturation  of  fielded  equipment. 

The  Center  would  have  primary  responsibility  for  managing  and 
coordinating  both  Stage  1  (Assessment)  and  Stage  2  (Implementation 
of  Improvements)  of  maturational  development. 

The  Center’s  primary  objectives  would  be  to 

•  Develop  advanced  plans  to  meet  future  needs. 

•  Formulate  guidance  for  Air  Force  and  industry  research. 

•  Review  the  allocation  of  resources  to  laboratory  programs  relat¬ 
ing  to  avionics. 

•  Manage  advanced  development  of  critical  components,  subassem¬ 
blies,  and  prototypes  for  future  subsystems. 

•  Manage  development  of  avionics  subsystems  that  are  applicable 
to  multiple  weapon  systems. 

•  Manage  development  of  avionics  subsystems  for  major  weapon 
systems  in  those  situations  where  the  weapon  system  SPO 
chooses  to  assign  management  responsibility  to  the  Center. 


•  Manage  R&M  maturation  programs  for 
Secondary  objectives  for  the  Center  would  be 
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selected  subsystems, 
to  assist: 

•  Weapon  system  development  programs. 

•  Product  improvement  programs. 

•  User  formulation  of  statements  about  forthcoming  needs. 

•  Development  of  advanced  concepts. 

•  Threat  assessments. 

As  envisioned  here,  the  already  sizable  Avionics  Laboratory  would  con¬ 
tinue  to  focus  on  advanced  research  and  would  operate  separately  from 
the  Avionics  Engineering  Center.  Although  operationally  separate,  the 
laboratory’s  planning  function  would  benefit  from  the  Center’s  plan¬ 
ning  activities.  Moreover,  the  Center  would  review  the  allocation  of 
funds  for  the  laboratory’s  advanced  research  programs.  To  minimize 
the  opportunities  for  conflicts  of  interest,  funds  for  advanced  research 
should  remain  separate  from  funds  that  the  Center  would  apply  to  its 
own  programs. 

To  accomplish  the  foregoing  objectives,  the  Center  would  need  to 
undertake  activities  within  four  major  areas: 

•  Field  assessment. 

•  Technology  management. 

•  Development  management. 

•  Planning. 

Field  Assessment  Area.  Activities  in  this  area  would  acquire, 
archive,  and  distribute  information  about  R&M  deficiencies  being 
experienced  in  the  field.  This  area  would  be  a  key  source  of  informa¬ 
tion  for  each  of  the  three  other  major  areas.  To  fully  accomplish  its 
purpose,  this  area  would  need  to  launch  and  support  efforts  aimed  at 
five  objectives: 

1.  Manage  data  collection  and  analysis.  Activities  that  would 
support  this  objective  include  managing  Stage  1  (Assessment)  of 
maturational  development  and  routinely  extracting  relevant 
information  about  field  problems  from  the  Air  Force’s  standard 
data  systems. 

An  initial  high  priority  activity  could  be  Stage  1  assessment  programs 
for  the  APG-68  radar  used  by  the  F-16  C/D  and  the  ALQ-131  Block  2 
electronic  countermeasures  pod.5 

5Of  the  candidates  identified  in  Sec.  Ill,  these  are  the  ones  that  have  recently  entered 
production,  hence  the  urgency  to  get  started  with  Stage  1. 
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2.  Archive  and  distribute  information.  Activities  that  would 
support  this  objective  would  include  distilling  lessons  learned 
from  Stage  1  maturational  development  programs,  archiving 
such  information,  and  distributing  it  in  an  appropriate  format. 
Distribution  should  include  SPOs,  contractors,  and  appropriate 
laboratory  programs.6 

An  initial  high  priority  activity  would  be  to  archive  and  distribute 
information  from  the  exploratory  radar  applications  discussed  in  Sec. 
IV. 

3.  Improve  data  systems  support.  Activities  here  would  include 
developing  and  supporting  special  data  collection  procedures  to 
support  Stage  1  applications  of  maturational  development,  and 
specifications  for  needed  improvements  to  Air  Force  standard 
data  systems  such  as  CAMS. 

Initial  high  priority  activities  would  be  to  establish  procedures  for  col¬ 
lecting  engineering  data  on  support  equipment  and  to  improve 
PORTER’S  analysis  procedures  to  support  applications  to  Stage  1 
maturational  development  activities. 

4.  Improve  application  of  test  and  evaluation  resources. 

Two  activities  are  essential  here:  improving  airborne  and 
ground-based  test  and  evaluation  resources  and  scheduling  avail¬ 
able  assets. 

High  on  the  priority  list  should  be  the  development  of  facilities  to 
evaluate  operational  radars  and  to  measure  environmental  parameters 
within  avionics  subsystems.  The  former  facility  could  support  a  radar 
evaluation  program  aimed  at  operational  evaluations  of  radar  effective¬ 
ness.7  The  latter  facility  could  support  the  development  of  essential 
databases  for  the  AVIP  program.8 

5.  Coordinate  field  assessment  programs.  Coordination  would 
need  to  occur  in  three  directions.  Lateral  coordination  would  be 
needed  for  Technology  Management,  Development  Management, 
and  Planning,  and  product  improvement  programs  to  assure  that 

6To  a  certain  degree,  the  Air  Force  Acquisition  Logistics  Center  is  doing  this  at  the 
weapon  system  level.  An  Avionics  Engineering  Center  would  be  expected  to  pursue  these 
activities  to  a  far  greater  depth. 

7Such  a  program  might  be  modeled  after  the  Air  Force's  Electronic  Warfare  Evalua¬ 
tion  Program  at  Eglin  Air  Force  Base. 

8See  Sec.  V  for  a  discussion  of  AVIP. 
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the  Field  Assessment  area  understands  the  needs  of  the  other 
areas  and  the  opportunities  to  contribute.  Upward  coordination 
would  be  required  to  secure  adequate  personnel  and  funding 
resources.  Internal  coordination  within  the  area  would  be 
required  to  schedule  resources  against  priority  needs. 

Technology  Management  Area.  The  purpose  of  this  area  would 
be  to  assimilate  information  about  evolving  threats,  R&M  deficiencies, 
and  emerging  technologies  and  use  such  knowledge  to  manage  the 
development  of  technology  from  basic  research  through  subsystem  pro¬ 
totypes.  This  area  would  use  information  from  the  Field  Assessment 
and  Planning  areas  and  indirectly  would  be  a  major  supplier  of  tech¬ 
nology  for  the  Development  Management  area.  To  fully  accomplish  its 
purpose,  this  area  would  need  to  launch  and  support  efforts  aimed  at 
four  objectives: 

1.  Evaluate  technology  development  programs.  Accomplish¬ 
ing  this  objective  requires  periodically  reviewing  technology 
development  programs  and  their  progress  in  light  of  needs — both 
performance  and  R&M — and  in  consideration  of  alternative 
approaches.  The  Air  Force’s  portfolio  of  such  programs  would 
need  to  be  evaluated  for  balance  across  four  major  divisions 
(basic  research,  critical  component  development,  advanced 
assemblies,  and  prototypes).  A  review  of  investment  balance— in 
light  of  needs — would  also  be  needed  within  each  division. 

An  initial  high  priority  should  be  assigned  to  a  review  of  programs  in 
the  critical  component  development  division,  with  emphasis  on  the 
needs  of  next  generation  fighter  airplanes. 

2.  Manage  selected  technology  development  programs.  This 
objective  is  aimed  at  important  programs  that — for  whatever 
reason — fall  outside  the  purview  or  interests  of  the  laboratory. 
An  example  of  such  a  program  might  be  the  development  of  crit¬ 
ical  components  as  a  prelude  to  a  full-scale  engineering  develop¬ 
ment  effort.  Another  example  might  be  the  development  of  a 
prototype  subsystem  or  assembly  such  as  an  antenna. 

Two  high  priority  activities  could  be  competitive  development  of  alter¬ 
native  designs  for  an  active  phase  array  element  (critical  component) 
and  competitive  development  of  alternative  designs  for  an  electroni¬ 
cally  scanned  antenna  based  on  such  elements  (Sec.  V). 

3.  Improve  test  and  evaluation  resources.  To  adequately 
evaluate  progress  made  by  technology  development  programs,  it 


94 


is  often  necessary  to  develop  or  acquire  special  resources  for  test 
and  evaluation.  Activities  here  would  provide  such  resources. 

4.  Coordinate  technology  development  programs.  As  with  the 
Field  Assessment  area,  this  area  aho  would  require  lateral, 
upward,  and  internal  coordination.  Such  coordination  is  espe¬ 
cially  crucial  to  assure  the  relevance  and  value  of  the  products  of 
technology  development  programs.  An  additional  dimension  of 
coordination  here  is  the  opportunity  to  help  guide  industry’s 
internal  research  and  development  efforts. 

Development  Management  Area.  The  purpose  of  this  area  would 
be  to  assimilate  information  about  evolving  threats,  R&M  deficiencies, 
and  developed  technologies  and  use  such  knowledge  to  help  manage 
avionics  development,  including  product  improvement.  In  addition  to 
the  airborne  equipment,9  the  scope  of  involvement  would  include 
ground  support  equipment,  maintenance  instructions  (TOs),  and  train¬ 
ing.  This  area  would  be  a  major  consumer  of  information  from  the 
three  other  major  areas:  Field  Assessment,  Technology  Management, 
and  Planning. 

To  fully  accomplish  its  purpose,  this  area  would  need  to  launch  and 
support  efforts  aimed  at  seven  objectives: 

1.  Formulate  and  maintain  development  guidelines.  Activi¬ 
ties  aimed  at  this  objective  include  development  and  mainte¬ 
nance  of  standards  (Military  Specifications)  and  processes 
(AVTP;  see  Sec.  V).  They  also  should  aim  at  appropriate  distri¬ 
bution  of  information  about  R&M  lessons  learned  in  related  pro¬ 
grams. 

2.  Assist  SPOs  in  managing  weapon  system  or  subsystem 
development.  Here  the  main  activity  would  be  supplying 
knowledgeable  engineers  to  support  SPO  programs,  especially 
program  reviews.  The  goal  would  be  to  supply  engineers  with 
experience  in  at  least  Field  Assessment  and  Technology 
Management. 

3.  Manage  any  avionics  subsystem  developments  not 
assigned  to  SPOs.  Management  of  development  for  certain 
avionics  subsystems  is  done  directly  by  the  Air  Force.  Those 
programs  not  assigned  to  SPOs  could  become  activities  within 
the  purview  of  this  objective. 

4.  Manage  and  coordinate  Stage  2  of  maturational  develop¬ 
ment.  The  main  activities  would  be  managing  and  coordinating 

9The  word  equipment  and  subsystem  are  always  intended  to  include  both  hardware 
and  software. 
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Stage  2  of  maturational  development.  For  a  given  subsystem,  an 
activity  would  manage  and  coordinate  the  implementation  of  the 
improvement  package  that  the  Air  Force  selects  from  the  Stage 
1  effort. 

Initial  high  priority  activities  here  would  be  coordinating  the  improve¬ 
ments  for  the  radars  examined  in  Sec.  IV.  Because  PMRT  has  already 
occurred  for  each  radar,  the  role  here  would  be  one  of  coordination 
rather  than  direct  management. 

5.  Systems  engineering  and  evaluation  support.  Here  there 
are  two  key  activities.  The  First  is  to  draw  upon  results  from  the 
Field  Assessment  area  to  evaluate  the  R&M  situation  with  a 
subsystem  of  interest  and  share  that  evaluation  with  the  cog¬ 
nizant  program  office.  The  second  is  to  assure  that  adequate 
resources  for  systems  engineering  are  applied  to  development  of 
interface  specifications  whenever  the  Air  Force  takes  on  the 
responsibility  for  management  of  subsystem  development. 

6.  Improve  test  and  evaluation  resources.  As  with  the  Tech¬ 
nology  Management  area,  to  have  the  necessary  test  and  evalua¬ 
tion  resources  adequately  available  may  require  special  develop¬ 
ment  and  acquisition  efforts.  Electronic  warfare  and  fighter  fire 
control  radar  equipment  are  two  classes  of  equipment  in  special 
need  of  such  facilities. 

7.  Coordinate  technology  management  programs.  As  with 
the  other  major  areas,  significant  coordination  of  funds,  priori¬ 
ties,  and  resources  would  be  required. 

Planning  Area.  The  purpose  of  this  last  major  area  would  be  to 
develop  plans  for  the  Center  based  upon  evolving  projections  of  the 
threat,  evolving  assessments  of  field  R&M  deficiencies,  and  emerging 
technologies. 

Prospective  Benefit  of  an  Avionics  Engineering  Center 

A  center  formed  along  the  lines  of  the  preceding  model  would  have 
the  opportunity  to: 

•  Sponsor  advanced  development  of  critical  elements. 

•  Start  FSED  early  for  critical  subsystems. 

•  Supervise  maturational  development  for  critical  subsystems. 

•  Oversee  post-PMRT  maturation  and  engineering  support. 

Sponsor  Advanced  Development  of  Critical  Elements.  Just  as 
the  Engine  SPO  has  sponsored  advanced  development  of  high-risk  and 
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technology-intensive  components  (such  as  gas  turbine  generators),  so 
also  could  an  Avionics  Engineering  Center  sponsor  advanced  develop¬ 
ment  of  similarly  complex  and  important  equipment  (major  electronic 
assemblies,  new  architectures,  digital  communication  protocols,  etc.). 

Start  FSED  Early  for  Critical  Subsystems.  Even  with  the 
benefit  of  advanced  development  of  high-risk  critical  elements,  some 
subsystems  are  sufficiently  complex  that  they  would  also  benefit  from 
starting  full-scale  engineering  development  in  advance  of  the  airframe. 
Even  the  most  sophisticated  avionics  equipment  usually  starts  FSED 
after  the  airframe  and  engine.  And  once  avionics  FSED  does  start,  it 
usually  occurs  without  benefit  of  advanced  development  of  its  critical 
components.  This  practice  made  sense  when  combat  airplanes  con¬ 
sisted  primarily  of  airframes  and  engines  and  when  avionics  equipment 
was  added  after  designs  had  largely  been  completed.  Now,  however, 
avionics  equipment  plays  a  more  central  role  in  combat  performance, 
and  it  accordingly  overshadows  most  other  equipment  in  cost,  weight, 
volume,  and  complexity.  It  thus  needs  the  early  developmental  atten¬ 
tion  customarily  given  airframes  and  engines. 

Supervise  Maturational  Development  for  Critical  Subsys¬ 
tems.  The  engine  SPO  already  has  a  Component  Improvement  Pro¬ 
gram  aimed  at  maturing  engines.  An  Avionics  Engineering  Center 
should  have  similar  responsibility  not  only  for  new  avionics  equipment 
but  also  for  avionics  equipment  already  in  the  field. 

Oversee  Post-PMRT  Maturation  and  Engineering  Support. 
An  Avionics  Engineering  Center  could  be  responsible  for  avionics  sub¬ 
systems  both  before  and  after  PMRT.  Such  a  practice  would  enable  it 
to  draw  on  information  and  experiences  accumulated  before  PMRT  to 
help  oversee  post-PMRT  maturation  and  engineering  support.  In  addi¬ 
tion,  so-called  Avionics  Technical  Assistant  Contractors  could  provide 
not  only  technical  assistance  to  the  Avionics  Engineering  Center  but 
also  a  stable  base  for  retaining  corporate  memory.  To  ensure  objec¬ 
tivity,  such  contractors  would  have  no  contracts  with  the  government 
to  develop  hardware  or  software.  Technical  Assistant  Contractors  have 
been  used  for  years  in  the  aerospace  industry  and  more  recently  at  the 
AFSC  Armament  Division  at  Eglin  Air  Force  Base. 

With  the  foregoing  kinds  of  activities,  an  Air  Force  center  of 
engineering  excellence  for  avionics  could  lead  the  way  in  accelerating 
R&M-related  avionics  technologies  (Proposal  1),  improving  the  ability 
to  test  avionics  equipment  (Proposal  2),  providing  more  complete  feed¬ 
back  on  equipment  performance  (Proposal  3),  adopting  a  maintainabil¬ 
ity  indicator  (Proposal  4),  and  instituting  maturational  development 
(Proposal  5).  The  creation  of  an  Avionics  Engineering  Center  (Pro¬ 
posal  6)  therefore  is  the  linch  pin  for  a  cohesive  strategy  for  reforming 
avionics  acquisition  and  support. 


VII.  RECOMMENDATIONS  AND  CONCLUSIONS 


RECOMMENDATIONS 

The  recommendations  developed  in  Secs.  Ill  through  VI  are  sum¬ 
marized  here  in  the  form  of  a  set  of  related  proposals  that  constitute  a 
coherent  strategy  for  reforming  avionics  acquisition  and  support.  The 
strategy  concentrates  on  the  major  weaknesses  and  the  most  promising 
solutions  to  these  weaknesses  that  have  come  to  our  attention  during 
20  years  of  research  in  this  field.  The  strategy  is  organized  from  the 
viewpoint  that  R&M  is  built  upon  multiple  lines  of  defense  that  start 
with  research  on  basic  technologies  and  culminate  with  the  maturation 
of  specific  subsystems  and  their  support  equipment. 

Proposal  1:  Accelerate  R&M-Related  Avionics  Technologies 

The  first  line  of  defense  for  R&M  is  for  engineers  to  design  it  right 
from  the  outset.  However,  engineers’  ability  to  do  this  is  influenced 
greatly  by  the  accumulated  knowledge  regarding  the  capabilities  and 
limitations  of  the  technologies  used  in  a  design.  Unfortunately  to 
deliver  a  subsystem  design  with  the  specified  levels  of  functional  per¬ 
formance,  engineers  often  must  apply  the  latest  advances  in  technol¬ 
ogy.  For  emerging  technologies  that  are  critical  to  achieving  mission 
essential  performance,  it  is  therefore  essential  to  accumulate  a  body  of 
engineering  knowledge  as  rapidly  and  as  efficiently  as  possible  to  form 
a  solid  basis  from  which  design  may  proceed.  The  Air  Force  can  help 
strengthen  the  engineering  knowledge  available  to  designers  by 
accelerating  the  development  of  key  avionics  technologies.  Certain 
technologies,  moreover,  promise  especially  important  benefits  to  R&M. 
These  need  to  be  searched  out  and  emphasized.  To  help  do  this,  this 
proposal  has  three  parts: 

•  Accelerate  development  of  selected  functional  performance  tech¬ 
nologies. 

•  Accelerate  research  on  failure  pathologies. 

•  Accelerate  research  and  development  of  Built-in  Test  technolo¬ 
gies. 

Although  the  Air  Force  already  has  efforts  in  each  of  these  areas,  there 
are  opportunities  to  beneficially  increase  the  level  of  those  efforts. 

Accelerate  Development  of  Selected  Functional  Performance 
Technologies.  Several  avionics  development  efforts  that  are 
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important  to  combat  aircraft,  such  as  fighters,  will  be  entering  full- 
scale  engineering  development  during  the  next  five  to  ten  years.  To 
more  fully  prepare  for  these  important  design  efforts,  the  Air  Force 
should  review  the  current  state  of  critical  electronics  technologies  in 
search  of  opportunities  to  accelerate  the  pace  of  research  and  advanced 
development  where  appropriate.1  Following  such  a  review,  the  Air 
Force  could  reassess  the  adequacy  of  its  investment  in  these  areas  and 
then  tailor  both  the  level  and  composition  of  its  investments.  Candi¬ 
date  technologies  for  accelerated  research  and  development  include: 

•  Active  Phased  Array  Technology.  Some  new  technologies 
promise  to  greatly  improve  R&M  and  thereby  reduce  the  time 
and  expense  of  a  maturational  development  phase.  One  such 
technology  for  radars  is  an  active  phased  array  that  can  provide 
an  electronically  scanned  antenna. 

•  Gallium  Arsenide  Technology.  This  technology  promises  to 
expand  the  analog  processing  capacity  on  board  combat  airplanes 
the  way  VHSIC  is  expected  to  revolutionize  digital  processing. 
Because  this  technology  is  a  more  recent  development,  engineer¬ 
ing  knowledge  of  this  technology  is  still  evolving.2 

Accelerate  Research  on  Failure  Pathologies.  During  the  next 
decade,  the  Air  Force  will  use  technologies  like  VHSIC  to  expand  the 
functional  capacities  of  new  and  existing  combat  airplanes.  The  mag¬ 
nitude  of  this  investment  and  the  extent  of  this  reliance  on  such  tech¬ 
nologies  make  it  very  worthwhile  for  the  Air  Force  to  aggressively 
sponsor  research  into  the  failure  modes  for  these  technologies.3 

Accelerate  Research  and  Development  of  Built-in  Test  Tech¬ 
nologies.  Here  the  Air  Force  has  two  especially  important  opportuni¬ 
ties:  improvement  of  methods  and  techniques  and  exploitation  of 
VHSIC. 

!The  Air  Force  could  assess  ways  to  accelerate  development  of  avionics  technologies 
to  allow  more  time  for  learning  about  the  conditions  under  which  technology-peculiar 
faults  will  arise — both  Type  A  and  Type  B  faults.  Such  efforts  would  complement  the 
maturational  development  process  for  the  selected  avionics  equipment  that  undergoes  the 
process  and  would  also  help  improve  the  R&M  characteristics  of  equipment  not  subjected 
to  the  maturational  development  process. 

2  At  this  point  designers  need  a  better  engineering  database  to  support  the  selection  of 
operating  temperatures  that  achieve  appropriate  balance  between  reliability  benefits  and 
design  penalties  in  terms  of  the  size,  weight,  and  power  required  by  the  environmental 
control  system. 

3New  technologies  such  as  VHSIC  and  gallium  arsenide  circuits  often  raise  great 
expectations  about  improved  reliability  and  even  about  invulnerability  to  potentially  seri¬ 
ous  failures.  Time  and  again,  development  programs  have  fallen  victim  to  what  might  be 
termed  the  “Titanic  Syndrome.”  The  Air  Force  can  avoid  catastrophic  R&M  “icebergs” 
only  by  improving  our  knowledge  in  advance  about  failure  pathologies  for  critical  new 
technologies. 
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•  Methods  and  Techniques.  An  initial  step  could  be  the 
immediate  launching  of  a  special  high-priority  project  to  assess 
and  accelerate  the  advancement  of  BIT  technology.  Such  a  proj¬ 
ect  could 

—  Review  current  BIT  mechanizations  and  catalog  approaches, 
strengths,  and  weaknesses. 

—  Assess  the  major  weaknesses  in  BIT  technology  and  in  con¬ 
temporary  applications  of  that  technology. 

—  Develop  a  research  plan  to  further  the  technology. 

—  Institute  guidelines  for  testing  BIT  and  for  applying  BIT 
technology  during  hardware  and  software  design. 

•  Exploitation  of  VHSIC.  VHSIC  technology  has  considerable 
untapped  potential  for  improving  subsystem  level  BIT.  Increased 
research  in  this  area  could  pay  large  dividends.4 

By  selectively  accelerating  the  advancement  of  key  R&M-related 
technologies  the  Air  Force  can  best  strengthen  the  engineer’s  ability  to 
design  the  equipment  right  from  the  outset.  This  is  the  first  line  of 
defense  for  R&M.  However,  even  the  best  of  designs  will  eventually 
develop  faults  during  operational  service.  The  second  line  of  defense, 
comprehensive  and  accurate  testing  for  faults,  is  the  subject  of  the  next 
proposal. 

Proposal  2:  Improve  the  Ability  to  Test  Avionics  Equipment 

In  the  airplane,  on  the  flightline,  in  the  shop,  and  at  the  depot,  the 
Air  Force  needs  improved  abilities  to  test  avionics.  Even  with  the  best 
of  efforts  to  accelerate  the  advancement  and  application  of  R&M  tech¬ 
nologies,  faults  will  inevitably  develop  in  avionics  equipment.  As  a 
consequence,  the  ability  to  test  and  find  faults  is  essential  to  maintain¬ 
ing  designed  capabilities. 

Improve  Ongoing  Programs.  To  improve  the  testability  of 
avionics  equipment,  the  Air  Force  can  place  greater  emphasis  on 
improving  fault-isolation  capabilities  in  such  research  and  development 
programs  as  Ultra-Reliable  Equipment,  Modular  Avionics,  and  Avion¬ 
ics  Integrity. 

•  Ultra-Reliable  Equipment  Program.  Developing  subsystems 
with  ultra-high  levels  of  reliability  (2,000  to  10,000  hours  between 


4One  promising  idea  that  warrants  increased  attention  is  to  use  VHSIC  to  capture 
better  information  about  the  operating  state  and  environment  of  avionics  equipment 
when  faults  are  detected. 
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failure)  is  desirable  as  long  as  the  subsystems  are  practical.5 
However,  projects  exploring  ways  to  build  ultra-reliable  equip¬ 
ment  need  to  ensure  that  their  labors  do  not  ignore  maintainabil¬ 
ity,  especially  because  current  fault- isolation  systems  fail  to  con¬ 
sistently  identify  the  more  difficult  and  more  elusive  Type  B 
faults  that  plague  current  weapon  systems.6 

•  Modular  Avionics  Program.  The  packaging  of  avionics  equip¬ 
ment  needs  to  be  examined  in  light  of  the  high  costs  of  flightline 
replaceable  units,  some  of  which  approach  $1  million.  One  alter¬ 
native  currently  drawing  much  interest  would  have  flightline 
technicians  remove  modules — less  expensive  and  about  the  same 
size  as  a  current  circuit  card— directly  from  the  airplane.  This 
seemingly  attractive  idea  may  increase  the  likelihood  of  disrupt¬ 
ing  delicate  connections.  Moreover,  because  current  BITs  often 
cannot  consistently  isolate  faults  to  specific  LRUs  (especially  for 
Type  B  faults)  research  on  modular  avionics  must  place  great 
emphasis  on  improving  the  fault-isolation  capabilities  of  BITs.7 

•  Avionics  Integrity  Program.  The  Air  Force  has  been  consid¬ 
ering  a  plan  for  an  Avionics  Integrity  Program  patterned  after  its 
Structural  and  Engine  Integrity  Programs.  These  programs  aim 
at  helping  designers  choose  materials  that  avoid  undue  risks  of 
catastrophic  failures  or  unaffordable  maintenance  burdens.  The 
Avionics  Integrity  Program  can  potentially  help  designers  most 
by  measuring  thermal  and  dynamic  stresses  placed  on  electronic 
assemblies  during  routine  operations,  but  it  must  not  ignore 
improving  fault-isolation  capabilities  in  general  and  BIT  capabili¬ 
ties  in  particular. 


5They  must  not  become  unaffordable,  too  large  and  too  heavy  for  a  combat  airplane, 
too  demanding  in  their  cooling  requirements,  or  too  hard  to  fix  when  they  fail.  Some 
proponents  believe  that  ultra-high  levels  of  reliability  (with  fighter  radars  enjoying  a 
2000-hour  mean  time  between  confirmed  failure)  are  both  affordable  and  achievable 
given  the  current  state  of  the  art.  Further,  some  believe  that  such  reliability  levels  jus¬ 
tify  decreased  emphasis  on  BIT.  Others  strongly  disagree.  Given  the  complexity  of  the 
equipment  and  the  disagreement  about  realizable  levels  of  reliability,  the  Air  Force 
should  strive  for  ultra-high  reliability  only  to  the  extent  that  it  does  not  short-change  the 
maintainability  that  will  be  needed  for  realizable  levels  of  reliability. 

6There  is  a  further  concern  that  the  trend  toward  micro-miniaturization  may  appreci¬ 
ably  increase  the  density  and  number  of  pins  on  a  typical  circuit  card.  Such  an  outcome 
would  further  increase  the  risk  of  faulty  connections  that  can  be  a  source  of  Type  B 
faults. 

7There  is  disagreement  about  whether  adequate  BITs  and  sufficiently  robust  connec¬ 
tions  can  be  designed  for  the  flightline  environment.  A  shortfall  in  either  area  would 
adversely  affect  a  fighter  airplane  that  depends  on  quick  isolation  and  con  ‘ion  of 
faults. 
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Improve  Capabilities  of  Tests.  For  ground-based  test  equipment, 
the  Air  Force  can  improve  the  capabilities  of  tests  by  developing  test 
translation  dictionaries,  direct  entry  into  test  sequences  for  specific 
sections  of  lengthy  ground  avionics  tests,  loop  testing  for  specific  tests, 
and  special  environmental  and  system  bench  capabilities  for  depots. 

•  Test  translation  dictionaries.  Such  dictionaries  would  enable 
avionics  technicians  at  one  maintenance  level  to  translate  test 
results  from  another  maintenance  level  into  terms  they  will  find 
useful  for  isolating  and  correcting  faults. 

•  Direct  entry  into  test  sequences  for  specific  sections  of 
lengthy  ground  avionics  tests.  Such  direct  entry  would  enable 
technicians  to  avoid  having  to  run  tests  in  an  invariable, 
predetermined  sequence.  As  a  consequence,  technicians  would 
have  greater  ability  to  find  Type  B  faults  that  are  sensitive  to 
time-varying  thermal  conditions. 

•  Loop  testing  for  specific  tests.  Such  loop  testing  would 
enable  technicians  to  run  the  same  test  repeatedly.  As  a  conse¬ 
quence,  it  improves  the  prospects  of  catching  certain  Type  B 
faults. 

•  Special  environmental  and  system  bench  capabilities  for 
depots.  Such  capabilities  would  enable  test  equipment  to  better 
replicate  operational  modes  and  environmental  settings  that 
especially  influence  Type  B  faults. 

Proposal  3:  Provide  More  Complete  Feedback  on  Equipment 

Even  with  the  best  of  test  capabilities,  some  faults  will  inevitably 
escape  detection  by  ground  support  equipment.  Thus,  R&M  requires  a 
third  line  of  defense:  Maintenance  personnel  need  timely  and  reason¬ 
ably  complete  feedback  to  deal  quickly  and  effectively  with  faulty 
assets  that  escape  repair. 

To  better  provide  such  information,  the  Air  Force  can  improve  the 
quality  of  information  received  from  the  pilot’s  postflight  debriefing  to 
maintenance  technicians,  and  it  can  improve  the  technicians’  capability 
to  track  and  identify  hard- to- fix  faults. 

Improve  Quality  of  Information  Received  by  the  Pilot 
Debriefing.  The  Air  Force  can  explore  improving  the  quality  of  pilot 
debriefings  by  using  an  automated  system  that  would  provide  a  menu 
of  questions  concerning  airplane  malfunctions.  Such  a  system  could 
use 


Data  transfer  units  to  capture  information  from  the  BIT. 


102 


•  Personal  computers  to  record  information  interactively  from 
pilots. 

These  measures  would  help  solve  the  very  difficult  problem  of  obtain¬ 
ing  sufficient  information  about  symptoms  and  operating  conditions  for 
Type  B  faults. 

Improve  the  Tracking  and  Correction  of  R&M  Deficiencies. 

To  improve  the  tracking  and  correction  of  R&M  deficiencies,  the  Air 
Force  could  institute  a  capability  to  do  Performance  Oriented  Tracking 
of  Equipment  Repair  (PORTER),  an  experimental  prototype  system 
using  personal  computers  to  track,  identify,  and  help  correct  faulty 
equipment.  PORTER  aims  at  reducing  avionics  maintenance  burdens, 
especially  for  sophisticated  equipment  such  as  fire  control  radars, 
through  judicious  collection  and  timely  transmission  of  performance 
and  maintenance  information  to  critical  points  in  the  support  process. 
This  information  should  enable  maintenance  personnel  to  identify  and 
more  quickly  fix  equipment  that  otherwise  would  circulate  repeatedly 
through  the  support  process,  using  up  time  and  scarce  resources. 

Proposal  4:  Adopt  a  Maintainability  Indicator 

Even  with  the  best  of  feedback  to  maintenance  technicians  about 
problems  with  equipment  performance,  R&M  needs  a  fourth  line  of 
defense  to  attract  needed  management  attention  to  resolve  the  more 
serious  maintainability  problems.8  A  single  measure  to  indicate  the 
overall  maintainability  of  a  subsystem  and  its  associated  ground  sup¬ 
port  system  would  be  desirable.  Such  an  indicator  could: 

•  Complement  the  existing  reliability  indicator  (MTBF)  and 
together  with  it  provide  a  meaningful  composite  picture  of  equip¬ 
ment  R&M. 

•  Be  sensitive  to  the  full  range  of  problems  that  arise  in  identifying 
faults  and  isolating  their  causes. 

•  Account  for  all  flights  with  indications  of  faulty  subsystem  opera¬ 
tion. 

The  following  proposed  maintainability  indicator  is  consistent  with 
these  principles: 


8Such  attention  is  essential  to  identifying  and  fixing  the  root  causes  of  problems.  It  is 
also  essential  for  effectively  communicating  maintainability  problems  to  the  research  and 
development  community.  Such  communication  is  essential  to  avoiding  repetition  of 
maintainability  problems  in  the  development  of  new  equipment. 
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Fault 

MTBI 

Removal 

x  100% 

Efficiency 

MTBF 

where  MTBI 

=  Mean  flying  Time  Between  flights  with  one  or 
more  Indications  of  faulty  operation  of  the  avionics 
subsystem. 

MTBF 

=  Mean  flying  Time  Between  flights  with  a  shop 
confirmed  Failure  of  the  avionics  subsystem. 

A  fault  removal  efficiency  of  100  percent  means  that  for  every  flight 
during  which  a  subsystem  manifested  a  symptom  of  degraded  perfor¬ 
mance,  technicians  removed  a  fault  from  that  subsystem  before  the 
next  flight.  A  25  percent  efficiency  means  that  an  average  of  four 
flights  with  symptoms  occurred  before  technicians  removed  a  fault. 

Proposal  5:  Institute  Maturational  Development 

Concept.  Even  with  the  best  of  implementations  for  the  preceding 
proposals,  some  important  R&M  problems  will  inevitably  evade  early, 
satisfactory  resolution,  especially  Type  B  faults  in  large,  complex,  and 
tightly  integrated  subsystems  that  incorporate  many  new  technologies. 
Fire  control  radars  for  fighter  airplanes  fall  in  this  category.  Such  sub¬ 
systems  and  their  associated  ground  support  systems  need  a  develop¬ 
ment  phase  to  mature  their  R&M  to  the  levels  that  will  allow  the  sub¬ 
systems  to  regularly  deliver  the  full  measure  of  performance  for  which 
they  were  designed.  Such  a  maturational  development  phase  needs  two 
stages: 

•  Stage  1  (Assessment):  Collection  and  analysis  of  engineering 
data  while  the  subsystem  is  in  normal  use  by  the  operator,  fol¬ 
lowed  by  analysis  of  candidate  improvements  and  formulation  of 
a  comprehensive  package  of  improvements.9 

•  Stage  2  (Implementation):  Implementation  of  the  most  cost- 
effective  improvements  that  aim  at  regular  delivery  of  full  design 
performance. 

Such  a  process  offers  a  further  line  of  defense  that  is  needed  to  assure 
the  delivery  of  necessary  R&M  characteristics  for  the  most  complex 
avionics  subsystems  and  would  be  an  important  supplement  to  the 
measures  suggested  by  the  foregoing  proposals. 

9  Are  as  of  needed  improvement  may  include  airborne  equipment,  ground  support 
equipment,  hardware,  software,  and  maintenance  procedures. 
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Implementation.  The  Air  Force  can  complete  the  implementation 
stage  for  ongoing  efforts  to  mature  the  F-15  and  F-16  radars,  it  can 
start  data  collection  and  analysis  stages  for  selected  other  equipment, 
and  it  can  institute  a  formally  planned  maturational  development 
phase  for  avionics  equipment  on  new  airplanes: 

•  Complete  the  Implementation  Stage  for  Ongoing  Efforts  to 
Mature  the  F-15  and  F-16  Radars.  The  Air  Force  is  demon¬ 
strating  maturational  development  on  the  F-15C/D  APG  63  and 
the  F-16A/B  APG  66  radars,  as  a  result  of  the  F-15/F-16  Radar 
R&M  Improvement  Program.  Integrated  sets  of  improvements 
have  been  defined,  evaluated,  and  briefed  by  the  cognizant  Air 
Force  organizations,  including  the  Aeronautical  Systems  Division 
(ASD)  Strike  SPO  that  managed  the  radar  contractors’  efforts. 
Unfortunately,  management  responsibility  for  these  improve¬ 
ments  has  since  been  diffused  through  many  organizations.  The 
Air  Force’s  current  challenges  are  (1)  to  continue  its  effective 
support  for  efforts  to  improve  the  R&M  of  the  F-15  and  F-16 
radars  and  (2)  to  demonstrate  its  commitment  both  to  matura¬ 
tional  development  as  a  concept  and  to  R&M  as  a  major  goal. 

•  Start  Assessment  Stage  for  Selected  Other  Equipment.  In 
addition  to  the  F-15  C/D  and  F-16  A/B  radars,  other  avionics 
equipment  can  derive  comparable  benefits  from  maturational 
development.  This  equipment  includes  radars  (the  APG  68  for 
the  F-16  C/D,  the  APG  70  for  the  F-15E),  ECM  equipment, 
weapon  delivery  systems,  and  pods  such  as  those  forming  the 
LANTIRN  and  targeting  subsystem.10 

•  Institute  a  Formally  Planned  Maturational  Development 
Phase.  Because  it  costs  extra  money  and  runs  the  risk  of  creat¬ 
ing  undesirable  publicity  about  R&M  problems,  some  weapon 
system  development  programs  may  be  reluctant  to  add  a  matura¬ 
tional  development  phase  to  their  development  efforts.  To 
minimize  such  reluctance,  a  maturational  development  phase 
could  be  institutionalized  as  a  formal  and  preplanned  part  of  each 
program’s  management  plan. 

Maximizing  the  Benefit.  Generally,  maturational  development 
will  offer  the  greatest  rewards  for  new  equipment  that  enters  develop¬ 
ment  early  enough  to  allow  one  or  more  follow-on  development  efforts 
for  maturation  prior  to  high-rate  production.  Such  early  application  is 

l0The8e  subsystems  have  been  selected  because  they  are  nearly  as  complex  as  the  two 
radars  already  subjected  to  the  data  collection  and  analysis  process.  Moreover,  these 
subsystems,  like  the  two  radars,  depend  on  leading  edge  technologies  that  may  still  be 
experiencing  growing  pains. 
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what  we  call  Approach  A  to  maturational  development.  It  is  the  pre¬ 
ferred  approach  in  terms  of  maximizing  the  opportunity  to  avoid  the 
high  costs  of  retrofitting  hardware.  Approach  B  pertains  to  situations 
where  avionics  subsystems  do  not  benefit  from  an  early  start  to  their 
full-scale  engineering  development.  Opportunities  for  beneficial  appli¬ 
cation  of  Approach  B  include  situations  where  already  fielded  equip¬ 
ment 

•  And  its  associated  support  equipment  are  struggling  to  depend¬ 
ably  deliver  designed  levels  of  mission  essential  performance 
because  of  R&M  difficulties. 

•  And/or  its  support  equipment  are  going  to  receive  significant 
performance-oriented  improvements. 

Although  Approach  B  may  prove  to  be  the  predominant  approach, 
experiences  from  the  exploratory  applications  of  maturational  develop¬ 
ment  have  shown  that  there  is  considerable  value  to  be  gained  by 
speeding  up  the  acquisition  and  development  process  to  start  avionics 
development  earlier.  The  typical  late  start  leads  to  a  hurried  develop¬ 
ment  effort  that  yields  little  time  to  explore  the  implications  of  using 
new  technologies  and  materials.11 

Late  start  and  early  termination  of  engineering  are  in  turn  com¬ 
pounded  by  transition  of  engineering  management  responsibilities  from 
the  SPO  to  the  AFLC  long  before  the  equipment  has  matured.  This 
transfer  of  engineering  responsibility  generally  occurs  during  the 
equipment’s  early  operational  life,  a  time  when  engineers  assigned  to 
the  SPO  are  only  beginning  to  identify  the  equipment’s  strengths  and 
weaknesses.  Once  the  AFLC  System  Program  Manager  assumes 
responsibility,  a  new  and  much  smaller  group  of  Air  Force  engineers 
becomes  responsible  for  the  equipment’s  further  maturation.12  More¬ 
over,  with  an  impending  transfer  of  R&M  engineering  responsibilities, 
it  is  hard  to  keep  a  development  organization  enthusiastically  engaged 
in  improving  R&M.  Such  considerations  support  the  idea  of  starting 
full-scale  engineering  development  of  avionics  soon  enough  to  allow 
time  for  the  preponderance  of  maturation  to  occur  before  PMRT. 

uThis  late  start  has  been  compounded  by  prematurely  terminating  the  involvement 
of  engineers.  Even  without  the  late  start,  merely  the  complexities  of  the  equipment  and 
its  multi-layered  support  process  guarantee  that  complex  R&M  problems  will  accompany 
the  equipment  when  it  enters  operational  service.  At  this  juncture,  an  engineering  data¬ 
base  and  analysis  of  the  situation  are  critical  to  resolving  dominant  problems.  Engineer¬ 
ing  involvement  at  this  point  has  been  far  too  shallow  to  build  the  kind  of  R&M  data¬ 
base  and  associated  analysis  that  the  F-15/F-16  Radar  R&M  Improvement  Program 
found  so  essential. 

12In  addition,  these  engineers  must  initiate  entirely  new  contracting  documents  for 
any  redesign  work.  This  contracting  procedure  is  far  more  burdensome  than  the  one  the 
SPO  follows  to  accomplish  the  same  sort  of  design  changes  before  PMRT.  Thus,  any 
improvements  in  equipment  maturation  slow  considerably  after  PMRT. 
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Even  with  an  early  start,  the  maturation  process  is  costly  and  time 
consuming.  Thus,  a  cost  effective  strategy  for  improving  R&M  must 
include  additional  lines  of  defense  decreasing  the  problems  that  need  to 
be  cleaned  up  during  a  maturational  development  phase.  To  that  end, 
the  strategy’s  first  four  proposals  aim  to  reduce  the  residual  set  of 
problems  that  would  need  to  be  addressed  by  maturational  develop¬ 
ment  (Proposal  5).  The  sixth  and  final  proposal  addresses  an  oppor¬ 
tunity  to  amplify  the  benefits  the  Air  Force  can  reap  from  the  first  five 
proposals. 

Proposal  6:  Reorganize  Avionics  Engineering  Resources 

For  complex  advanced  technology  subsystems,  multiple  lines  of 
defense  must  be  called  upon  to  achieve  needed  levels  of  excellence  in 
R&M.  By  reorganizing  its  avionics  engineering  resources  the  Air  Force 
can  better  posture  itself  to  coordinate  defenses  in  a  synergistically 
beneficial  manner.  R&M-related  technologies  (especially  BIT  technol¬ 
ogy)  need  to  be  advancing  in  support  of  development  programs  that 
improve  test  capabilities,  while  advances  in  performance-oriented 
tracking  of  equipment  repair  need  to  be  coordinated  to  exploit 
improvements  to  the  type  of  information  that  new  test  technologies 
and  ongoing  development  programs  can  provide. 

In  contrast  to  such  finely  tuned  coordination,  past  avionics  research 
and  development  efforts  have  suffered  from  lack  of  (1)  R&M  guidance 
to  laboratory  projects,  (2)  a  potent  sponsor  for  the  advanced  develop¬ 
ment  of  critical  elements,  (3)  an  agency  dedicated  to  supervising 
maturational  development,  and  (4)  an  agency  with  a  robust  engineering 
organization  overseeing  post-PMRT  maturation  of  both  airborne  and 
ground  support  equipment. 

One  way  that  the  Air  Force  could  address  such  deficiencies  is  to 
reorganize  its  avionics  engineering  resources  in  the  form  of  what  we 
call  an  Avionics  Engineering  Center.  Such  a  center  would  help 13  over¬ 
see  research,  development,  and  maturation  of  sophisticated  avionics 
subsystems  (fire  control  radars,  electronic  warfare  systems,  and  the 
like)  by 

1.  Sponsoring  Advanced  Development  of  Critical  Elements. 

An  Avionics  Engineering  Center  could  sponsor  advanced 
development  of  complex  and  important  equipment  (such  as 
major  electronic  assemblies,  new  architectures,  and  digital  com¬ 
munication  protocols). 

l3Primary  responsibility  for  managing  subsystem  development  may  reside  with  a 
weapon  system’s  prime  contractor,  or  its  System  Program  Office  or  an  Avionics 
Engineering  Center.  In  the  first  two  situations,  the  role  of  the  Center  would  be  to  assist 
the  cognizant  SPO. 
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2.  Starting  FSED  Early  for  Critical  Subsystems.  Even  with 
the  benefit  of  advanced  development  of  high-risk  critical  ele¬ 
ments,  some  subsystems  are  sufficiently  complex  that  they 
would  also  benefit  from  starting  FSED  in  advance  of  the  air¬ 
frame. 

3.  Supervising  Maturational  Development  for  Critical  Sub¬ 
systems.  An  Avionics  Engineering  Center  could  have  responsi¬ 
bility  not  only  for  selected  new  avionics  equipment  but  also  for 
selected  avionics  equipment  already  in  the  field.14 

4.  Overseeing  Post-PMRT  Maturation  and  Engineering  Sup¬ 
port.  An  Avionics  Engineering  Center  could  be  responsible  for 
avionics  subsystems  both  before  and  after  PMRT.  In  addition, 
avionics  technical  assistant  contractors  could  provide  not  only 
technical  assistance  to  the  Avionics  Engineering  Center  but  also 
a  stable  base  for  retaining  corporate  memory. 

In  the  past  there  has  been  resistance  to  such  concepts  as  an  Avionics 
Engineering  Center  and  subsystem-focused  maturational  development. 
Commonly  cited  reasons  include  concerns  about  the  adequacy  of  the 
Air  Force’s  resources  that  could  be  allocated  to  managing  subsystem 
development  and  maturation,  especially  in  areas  with  complex  inter¬ 
faces.  Other  concerns  are  based  upon  fears  that  the  application  of 
technological  advances  may  be  retarded  and  the  prime  contractors’ 
ability  to  optimize  a  weapon  system  may  be  seriously  constrained. 
Against  such  concerns  the  Air  Force  needs  to  weigh  two  considerations. 
First  is  the  relationship  between  past  practices  and  current  difficulties 
with  avionics  R&M.  Second  is  the  new  challenges  that  will  flow  from 
the  rapidly  changing  character  of  avionics  technology. 


l4To  ensure  objective  management  of  such  an  effort,  the  group  that  manages  the  data 
assessment  effort  could  be  independent  of  the  group  responsible  for  the  subsystem’s 
development.  While  independent,  both  groups  could  reside  within  an  Avionics  Engineer¬ 
ing  Center  or  within  a  weapon  system  SPO.  In  the  case  of  the  Radar  R&M  Program, 
such  independence  proved  very  beneficial.  Although  certain  tensions  were  created,  they 
stimulated  creativity  and  sharpened  a  focus  on  important  problems  and  critical  uncer¬ 
tainties.  Such  a  center  could  also  help  in  preplanning,  which  is  another  key  to  formaliz¬ 
ing  the  maturation  process.  Certain  types  of  equipment  have  a  high  enough  likelihood  of 
benefiting  from  the  process  that  the  program  management  plan  for  development  could 
beneficially  specify  time  and  funds  for  the  Stage  1  assessment  effort.  Such  an  up-front 
commitment  could  save  time  and  funds.  Up  to  two-thirds  of  the  cost  for  implementing 
the  R&M  improvements  for  the  F-15  C/D  radar  and  the  F-16  A/B  radar  probably  could 
have  been  avoided  by  promptly  executing  such  an  R&M  maturation  process  immediately 
upon  the  equipment's  entry  into  operational  service. 
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CONCLUSIONS 

For  its  combat  airplanes  to  meet  and  defeat  future  threats,  the  Air 
Force’s  avionics  subsystems  will  continue  to  grow  in  terms  of  functions, 
sophistication,  and  complexity,  while  relying  heavily  on  the  latest 
advances  in  technology.  To  meet  the  challenge  of  acquiring  and  sup¬ 
porting  the  full  designed  capabilities  of  such  equipment,  the  Air  Force 
needs  to  aspire  to  high  levels  of  excellence  in  avionics  R&M.  The  Air 
Force  would  benefit  from  the  improved  coordination  that  an  Avionics 
Engineering  Center  could  provide  to  managing  multiple  lines  of 
defense.  Such  a  center  could  cap  a  major  reform  built  upon  four 
cornerstone  concepts: 

•  Explicit  recognition  of  Type  B  faults. 

•  Implementation  of  PORTER  to  more  rapidly  identify  Type  B 
problems. 

•  Use  of  fault  removal  efficiency  as  a  management  indicator  for 
maintainability. 

•  Institutionalization  of  maturational  development  as  a  last  line  of 
defense  for  R&M. 

Although  any  of  these  concepts  would  prove  beneficial,  the  complete 
strategy  would  constitute  a  major  reform  to  how  the  Air  Force  acquires 
and  supports  avionics.  With  such  reform,  the  Air  Force  could  exploit 
rapidly  advancing  electronics  technologies  to  achieve  levels  of  excel¬ 
lence  in  avionics  that  would  ensure  the  superiority  of  its  combat  air¬ 
planes. 


Appendix  A 


PRELIMINARY  ASSESSMENT  OF 
AVIONICS  MATURITY 


This  appendix  describes  a  preliminary  assessment  of  avionics  matu¬ 
rity  conducted  by  RAND  in  1980  using  the  Air  Force’s  standard  data 
sources  for  the  F-15  and  F-16.  Although  these  data  are  not  new,  they 
illustrate  the  limitations  of  standard  data  collection  systems,  and  con¬ 
clusions  about  avionics  maturity  drawn  from  them  have  been  largely 
substantiated  by  the  subsequent  special  data  collection  efforts 
described  in  the  text. 


LIMITATIONS  OF  STANDARD  DATA 
COLLECTION  SYSTEMS 

The  Air  Force’s  standard  data  collection  systems  have  a  limited  abil¬ 
ity  to  provide  conclusive  and  in-depth  views  of  the  nature  of  potential 
avionics  R&M  problems  and  the  extent  to  which  these  problems  may 
cause  the  avionics  equipment,  and  hence  the  aircraft,  to  deviate  from 
its  designed  level  of  performance. 

The  standard  data  collection  systems  can  best  deal  with  Type  A 
faults,  where  symptoms  are  observable  no  matter  who  examines  the 
equipment  and  no  matter  where  the  examination  occurs.  However, 
many  of  our  avionics  R&M  problems  result  from  Type  B  faults,  where 
symptoms  do  not  always  repeat  and  are  hard  to  pin  down.  Such  faults 
play  havoc  with  attempts  to  develop,  operate,  and  support  sophisticated 
avionics  equipment. 

Although  standard  data  collection  systems  collected  data  throughout 
and  subsequent  to  the  entire  development  and  production  process  for 
the  F-15  and  F-16,  these  data  failed  to  identify  many  of  the  Type  B 
faults  that  were  discovered  in  our  subsequent  special  data  collection 
efforts  (described  in  Sec.  IV). 

Even  by  1980 — over  four  years  after  the  F-15  had  achieved  IOC  and 
over  one  year  after  the  F-16  had  achieved  IOC — no  clear  picture  had 
evolved  concerning  whether  avionics  for  these  aircraft  had  matured  to 
the  point  of  regularly  delivering  their  full  designed  levels  of  perfor¬ 
mance. 
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Data  collected  during  1980  concentrate  on  the  number  of  LRUs  sent 
to  avionics  automatic  test  stations  per  flight  flown,  automatic  rather 
than  manual  test  stations  because  these  LRUs  tend  to  be  the  more 
sophisticated,  more  costly,  and  more  troublesome  ones  that  are  essen¬ 
tial  to  the  F-15’s  combat  effectiveness. 

The  most  accurate  source  of  data  on  these  LRUs  proved  to  be  the 
manual  forms  maintained  by  shop  technicians.1  Because  we  had  to 
review  forms  for  each  LRU  serial  number,  we  limited  our  inquiry  to  the 
more  sophisticated  LRU  types  processed  by  the  automatic  test  equip¬ 
ment.2 

Data  from  the  manual  forms  proved  much  more  accurate  than  data 
collected  by  the  Air  Force’s  computerized  maintenance  data  collection 
system  (known  as  MDC  or  AFTO  349). 3  Discussions  with  shop  and 
flightline  maintenance  personnel  elicited  the  following  plausible  expla¬ 
nations: 

•  The  shop  uses  manual  forms  (instead  of  the  AFTO  349  Form 
for  computerized  documentation)  to  document  specific  LRUs 
that  repeatedly  visit  the  shop  for  no  apparent  reason.  Shop 
personnel  place  highest  priority  on  these  manual  forms  because 
they  can  send  an  LRU  to  the  depot  after  it  has  visited  the  shop 
three  or  four  times. 

•  The  AFTO  349  Form  appears  unsuited  to  the  realities  of  avion¬ 
ics  maintenance.  It  relies  on  a  lengthy  list  of  codes  to  charac¬ 
terize  malfunctions  and  actions  taken,  and  it  fails  to  capture 
the  kind  of  information  that  maintenance  personnel  need  to  do 
their  jobs.  The  AFTO  349  Form  is  particularly  unsuited  to 
documenting  the  swapping  of  electronic  components  that  inevi¬ 
tably  occurs  on  the  flightline  and  in  the  avionics  shop  as  a 
last-ditch  means  of  isolating  faults  in  avionics  and  test  equip¬ 
ment.  Consequently,  separate  manual  devices,  such  as  the 
shop’s  LRU  form  and  log  books  on  the  flightline  and  in  the 
shop,  are  widely  used. 


'Air  Force  Technical  Order  Form  95. 

2Today,  at  Bitburg  Air  Base,  such  information  is  routinely  being  entered  into 
PORTER. 

3Air  Force  Technical  Order  Form  349  provides  the  input  for  the  Maintenance  Data 
Collection  (MDC)  System. 


PRELIMINARY  ASSESSMENT  OF  AVIONICS 
MATURITY  FOR  THE  F-15 
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Experiences  of  F-15s  in  the  49th  TFW  at  Holloman  AFB  in  1980 
strongly  suggested  areas  where  improved  maturity  of  avionics  subsys¬ 
tems  and  associated  support  resources  could  considerably  increase  the 
regular  delivery  of  full  designed  performance.  During  May,  June,  and 
July,  specialists  removed  660  LRUs  and  sent  them  to  the  avionics 
shop’s  automatic  test  stations.4  These  airplanes  averaged  one  such 
removal  for  each  five  flights  flown.  During  this  90-day  period,  250  of 
the  660  LRUs  sent  to  automatic  test  stations  were  “reoccurrences”: 
The  flightline  specialists  had  already  removed  the  same  LRU  type  at 
least  once  from  the  same  airplane  during  the  90-day  period.5 

Figure  A.l  groups  these  68  F-15s  into  quartiles  according  to  the  fre¬ 
quency  with  which  flightline  specialists  sent  LRUs  from  each  airplane 
to  automatic  test  stations.  Airplanes  in  the  third  quartile,  flying  27 
percent  of  the  wing’s  flights,  experienced  35  percent  of  the  wing’s 
removals,  with  roughly  one  occurring  for  every  five  flights.  Airplanes 
in  the  fourth  quartile,  flying  only  11  percent  of  the  wing’s  flights, 
experienced  38  percent  of  its  removals,  with  one  occurring  for  less  than 
every  two  flights.  Taken  together,  data  on  the  34  airplanes  in  these 
two  quartiles  indicate  that  half  of  the  wing’s  airplanes  averaged  almost 
one  removal  for  every  three  flights.  Moreover,  during  this  90-day 
period,  nearly  half  of  the  LRUs  sent  to  automatic  test  stations  from 
these  34  F-15s  were  reoccurring  removals. 

A  look  at  the  histories  of  some  particularly  troublesome  airplanes 
helps  identify  the  avionics  subsystems  where  improved  maturity  could 
increase  the  dependable  delivery  of  full  designed  capabilities.  Three 
F-15s — airplanes  7133,  7109,  and  7153 — accounted  for  41  of  the  250 
reoccurring  removals. 

4They  also  sent  a  comparable  quantity  of  other  LRUs  to  the  manual  test  stations. 
Automatic  stations  test  44  LRU  types,  manual  stations  56  types.  Excluded  from  our 
assessment  are  four  F-15s  that  hardly  flew  during  this  period.  As  this  report  is  being 
published  some  seven  years  after  these  data  were  collected,  the  overall  level  of  mainte¬ 
nance  activity  remains  roughly  the  same,  although  it  does  vary  from  base  to  base  and 
over  time  at  a  given  base. 

5We  use  the  term  “reoccurring”  to  avoid  confusion  with  “recurrent,”  whuii  'he  Air 
Force  uses  to  indicate  a  removal  that  recurs  within  three  flights,  and  “repeat,'  v,nith  the 
Air  Force  uses  to  indicate  a  removal  that  repeats  after  the  next  flight.  These  250  reoc¬ 
curring  removals  seem  excessive  for  so  short  a  time.  If  we  take  the  most  frequently 
removed  LRU  type  (1.7  removals  per  100  flights)  and  the  most  frequently  flying  of  these 
airplanes  (65  flights  during  the  90-day  period),  we  should  expect  only  1.1  removals  for 
the  most  frequently  removed  LRU  type.  To  assess  the  chances  that  these  reoccurring 
removals  were  the  result  of  a  simple  random  process,  we  statistically  tested  the 
hypothesis  that  these  reoccurring  removals  could  be  explained  as  the  outcome  of  a  simple 
Poisson  arrival  process.  The  chance  of  that  hypothesis  being  true  turned  out  to  be  less 
than  1  in  200. 
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Fig.  A.l— LRU  removals,  by  quartile,  for  the  49th  TFW 
(June  through  July  1980) 


Falling  in  the  third  quartile  with  49  percent  of  its  wing  experiencing 
more  LRUs  removed  per  flight,  Airplane  7133  had — as  Fig.  A.2 
shows — roughly  one  removal  for  each  six  flights  flown.  Its  removals 
occurred  in  the  head-up  display,  navigation,  and  radar  subsystems. 

Perhaps  most  important  are  this  airplane’s  experiences  with  its 
navigation  and  radar  subsystems.  Without  them,  the  F-15  can  fly  but 
it  cannot  carry  out  its  unique  combat  responsibilities.  In  the  naviga¬ 
tion  subsystem,  flightline  specialists  removed  and  replaced  the  inertial 
measurement  unit  three  times;  in  the  radar  subsystem,  they  removed 
and  replaced  the  receiver  three  times  and  the  analog  processor  four 
times.  These  experiences  indicate  more  than  a  frequent  removal  and 
replacement  of  LRUs:  They  suggest  that  during  the  month  of  June, 
while  the  airplane  was  flying  29  times,  it  probably  lacked  a  dependable 
fire  control  radar. 
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Fig.  A.2 — LRU  removals  from  F-15  number  7133 
(June  through  July  1980) 


Falling  in  the  fourth  quartile  with  respectively  16  and  ’0  percent  of 
their  wing  having  more  LRUs  removed  per  flight,  Airplanes  7109  and 
7153  had  even  more  frequent  removals.  As  Figs.  A.3  and  A.4  show, 
Airplane  7109  experienced  roughly  one  removal  for  each  two  flights 
and  Airplane  7153  one  for  each  flight.  Their  removals  occurred  in  the 
flight  controls,  inertial  navigation,  head-up  display,  radar,  and  instru¬ 
ments. 

Although  removals  appear  over  a  broad  spectrum,  they  seem  to  con¬ 
centrate  in  these  airplanes’  radar  and  for  Airplane  7153  in  its  inertial 
navigation  subsystem  as  well.  These  particular  removals  suggest  that 
Airplane  7109  probably  lacked  dependable  combat  avionics  equipment 
for  three  months,  during  which  it  flew  40  times,  and  that  Airplane  7153 
probably  lacked  it  for  two  months,  during  which  it  flew  29  times.  In 
addition,  the  histories  of  these  airplanes  suggest  that  flightline  special¬ 
ists  often  could  not  identify  malfunctioning  LRUs.  We  see  their  frus¬ 
tration  over  being  unable  to  isolate  faults  when  on  June  11  they 
removed  a  total  of  four  LRUs  from  Airplane  7103,  and  on  July  14  they 
removed  a  total  of  five  from  Airplane  7153,  in  both  cases  without  seem¬ 
ing  to  solve  the  problem. 
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Fig.  A.3—LRU  removals  from  F-15  number  7109 
(May  through  July  1980) 


PRELIMINARY  ASSESSMENT  OF  AVIONICS 
MATURITY  FOR  THE  F-16 

Experiences  of  F-16s  in  the  388th  TFW  at  Hill  AFB  in  1980  also 
strongly  suggested  areas  where  improved  maturity  of  avionics  subsys¬ 
tems  and  associated  support  resources  could  considerably  increase  the 
regular  delivery  of  full  designed  performance.  During  September, 
October,  and  November  1980,  104  of  its  F-16s  flew  4363  times,  and 
flightline  specialists  removed  799  LRUs  that  could  be  tested  by  the 
F-16’8  automatic  test  stations.6  These  airplanes  averaged  one  such 
removal  for  each  five  flights.  During  this  90-day  period,  of  the  799 
LRUs  sent  to  the  automatic  test  stations,  168  were  reoccurring  remov¬ 
als.  Although  repair  records  for  these  airplanes  show  fewer  reoccurring 
removals  than  those  experienced  by  F-15s,  this  situation  may  change 
once  the  wiring  and  connecters  on  F-16s  accrue  a  comparable  amount 
of  service  time. 

6 Automatic  stations  can  test  50  LRU  types  although  the  Air  Force  has  chosen  to  test 
only  37  of  them  at  the  airbase  level.  Statistics  on  the  F-16  are  computed  on  the  basis  of 
these  50  LRU  types. 
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Fig.  A.4 — LRU  removals  from  F-15  number  7153 
(June  through  July  1980) 

Figure  A.5  groups  these  104  F-16s  into  quartiles  according  to  the  fre¬ 
quency  with  which  flightline  specialists  sent  LRUs  from  each  airplane 
to  automatic  test  stations.  Airplanes  in  the  last  quartile,  flying  22  per¬ 
cent  of  the  wing’s  flights,  experienced  44  percent  of  the  wing’s  remov¬ 
als,  with  one  occurring  for  less  than  every  three  flights.  Moreover,  dur¬ 
ing  this  90-day  period,  roughly  one-fourth  of  the  LRUs  sent  to 
automatic  test  stations  from  these  26  F-16s  were  reoccurring  removals. 

Again,  a  look  at  the  histories  of  some  particularly  troublesome  air¬ 
planes  helps  identify  the  avionics  subsystems  where  improved  maturity 
could  increase  the  dependable  delivery  of  full  designed  capabilities. 
Three  F-16s — Airplanes  0079,  0021,  and  0086 — accounted  for  22  of  the 
168  reoccurring  removals. 

Falling  in  the  fourth  quartile  with  22  percent  of  its  wing  experienc¬ 
ing  more  LRUs  removed  per  flight,  as  Fig.  A.6  shows,  Airplane  0079 
had  roughly  one  removal  for  each  three  flights.  Its  removals  occurred 
in  the  flight  controls,  flight  control  instruments,  communications,  and 
radar  subsystems. 
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Fig.  A.5 — LRU  removals,  by  quartile,  for  the  388th  TFW 
(September  through  November  1980) 


Here  removals  appear  over  a  broad  spectrum  of  subsystems, 
although  most  occur  in  the  flight  controls  and  flight  control  instru¬ 
ments.  Flightline  specialists  on  October  26  removed  the  air  data  com¬ 
puter;  two  days  later  they  again  removed  the  air  data  computer,  this 
time  with  the  electronic  component  assembly;  two  days  later  they  once 
again  removed  the  electronic  component  assembly. 

Falling  in  the  fourth  quartile  with  16  and  15  percent  respectively  of 
their  wing  having  more  LRUs  removed  per  flight,  Airplanes  0021  and 
0068  had  even  more  frequent  removals.  As  Figs.  A.7  and  A.8  show, 
both  airplanes  experienced  roughly  one  removal  for  each  three  flights. 
Their  removals  occurred  in  the  flight  controls,  instruments,  radar, 
head-up  display,  fire  control  navigation,  electrical  optical  display, 
stores  management  set,  and  threat  warning  subsystems. 

Although  removals  again  appear  over  a  broad  spectrum,  they  seem 
to  concentrate  in  the  radar  subsystem  on  Airplane  0021  and  in  the  fire 
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Fig.  A.6 — LRU  removals  from  F-16  number  0079 
(September  through  November  1980) 


control  navigation  and  threat  warning  subsystems  on  Airplane  0086. 
These  particular  removals  suggest  that  Airplane  0021  probably  lacked 
dependable  combat  avionics  equipment  for  three  months,  during  which 
it  flew  50  times,  and  that  Airplane  0086  probably  lacked  full  combat 
effectiveness  for  more  than  two  months,  during  which  it  flew  38  times. 
The  history  of  Airplane  0021  also  suggests  that  flightline  specialists 
often  could  not  identify  its  malfunctioning  LRUs.  On  September  16 
they  removed  a  total  of  five  of  its  LRUs,  four  of  which  come  from  the 
radar,  without  seeming  to  solve  the  problem. 

The  quartile  analyses  for  the  F-15  and  the  F-16  reveal  very  broad 
distributions  in  terms  of  removals  per  flight.  Reoccurring  removals 
tend  to  happen  far  more  often  with  fourth  quartile  airplanes  than  with 
first  quartile  airplanes.  Because  fault  isolation  problems  give  rise  to 
reoccurring  removals,  the  quartile  analyses  show  the  extent  to  which 
such  maintainability  problems  are  degrading  the  condition  of  certain 
airplanes. 

Causes  of  Frequent  and  Reoccurring  Removals  of  LRUs 

Do  the  frequent  and  reoccurring  removals  of  LRUs  result  from  the 
immaturity  of  the  LRUs  and  the  test  equipment,  or  from  the  overwill- 


Fig.  A.7— LRU  removals  from  F-16  number  0021 
(September  through  November  1980) 


ingness  of  flightline  specialists  to  remove  apparently  troublesome 
LRUs  from  airplanes?7 

Our  research  in  1980  suggested  that  flightline  specialists  do  not 
remove  excessive  numbers  of  LRUs.  For  example.  Fig.  A.9  shows  that 
F-16  Number  0021  reported  on  14  occasions  between  September  1  and 
November  30,  1980  that  it  experienced  problems  with  its  radar.  Two  of 
these  reports  were  “Code  3,”  meaning  that  the  flightline  should 
immediately  correct  problems;  12  were  “Code  2,”  meaning  that  they 
could  delay  corrective  action.  Such  reports  indicate  two  things  about 
these  codes: 

1.  Many  peacetime  Code  2  reports  would  be  Code  3  reports  in 
wartime  because  pilots  could  not  tolerate  degraded  radar  sub¬ 
systems  during  combat. 


7In  F-15s  and  F-16s,  the  flightline  specialists  cannot  repair  LRUs,  they  can  merely 
send  them  to  the  avionics  shops. 


119 


2.  Flightline  specialists  do  not  always  respond  immediately  to 
Code  3  reports:  this  airplane  had  to  fly  seven  missions 
between  its  initial  Code  3  report  on  September  16  and  the 
removal  of  a  radar  LRU  on  September  22. 

In  spite  of  the  total  of  14  reports,  flightline  specialists  removed  a  radar 
LRU  on  only  six  occasions,  forcing  the  airplane  to  fly  at  least  67  flights 
between  reporting  a  faulty  radar  and  having  a  radar  LRU  removed. 
Overall,  in  1980  we  found  that  flightline  specialists  receive  roughly 
twice  as  many  complaints  as  they  remove  LRUs. 
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Fig.  A.8 — LRU  removals  from  F-16  number  0086 
(September  through  November  1980) 
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LIMITATIONS  OF  THE  1980  DATA 
COLLECTION  AND  ANALYSIS 

The  1980  data  collection  and  analysis  efforts  are  potentially  limited 
by  the  inferential  nature  of  the  analysis  and  the  dependability  of  the 
data. 

The  Inferential  Nature  of  the  Analysis.  The  analysis  inferred 
that  the  removal  of  an  LRU  meant  that  a  fault  was  present  and  that 
the  repeated  removals  of  LRUs  meant  that  one  or  more  faults 
remained  throughout  the  period  of  reoccurring  removals.  Although 
interviews  with  pilots,  maintenance  personnel,  and  contractor  field 
representatives  corroborated  the  general  reasonableness  of  these  infer¬ 
ences,  they  also  pointed  out  that  pilots  occasionally  request  mainte¬ 
nance  when  none  is  warranted,  and  technicians  occasionally  remove 
LRUs  when  there  are  no  defects.8  Because  of  the  complex  nature  of 


8Flightline  maintenance  personnel,  pilots,  and  experienced  contractor  engineering 
representatives  all  recount  anecdotes  about  the  other  two  who  do  not  seem  to  understand 
the  equipment. 


121 


fire  control  radars  and  because  radar  performance  can  be  influenced  by 
factors  external  to  the  aircraft,  there  is  no  way  of  knowing  how  often 
such  mistakes  occur. 

The  Dependability  of  the  Data.  Rapid  preparation  of  aircraft  for 
scheduled  flights  is  the  main  concern  of  flightline  maintenance  units. 
Accurate  documentation  is  not.  As  a  consequence,  maintenance  docu¬ 
mentation  does  not  always  accurately  record  the  airplane  number  from 
which  an  LRU  was  removed.  In  addition,  maintenance  personnel  may 
swap  LRUs  between  airplanes  to  facilitate  the  fault- isolation  process. 
Thus,  maintenance  documentation  may  record  the  last  airplane  the 
LRU  was  tried  in  on  the  ground  rather  than  the  airplane  in  which  it 
actually  flew. 
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BACKGROUND  FOR  EXPLORATORY 
APPLICATIONS  OF  MATURATIONAL 
DEVELOPMENT  TO  THE  F-15 
AND  F-16  RADARS 

The  preliminary  assessment  of  avionics  maturity  described  in 
Appendix  A  prompted  the  Directorate  for  Plans  at  the  Air  Force  Sys¬ 
tems  Command  (AFSC)  to  ask  RAND  to  define  and  help  implement  a 
demonstration  program  for  maturational  development.  The  program 
concept,  first  briefed  during  September  1980,  was  subsequently  briefed 
to  the  Air  Staff  during  June  1981.  The  proposed  plan  called  for  a 
two-stage  demonstration  program  involving 

•  Stage  1:  Data  collection  and  analysis  (including  definition  of 
promising  candidates  for  improvement). 

•  Stage  2:  Implementation  of  the  most  cost-effective  R&M 
improvement  options. 

As  requested  by  AFSC,  the  plan  concentrated  on  the  Westinghouse 
APG-66  block  15  radar  used  on  the  General  Dynamics  F-16  A/B  air¬ 
craft  and  the  Hughes  APG-63  radar  used  on  the  McDonnell  F-15C/D 
aircraft. 

According  to  the  plan,  Stage  1  would  consist  of  six-month  data  col¬ 
lection  and  analysis  efforts  at  Hahn  Air  Base  and  Hill  AFB  for  the 
F-16  and  at  Bitburg  Air  Base  and  Langley  AFB  for  the  F-15.  These 
efforts  would  be  conducted  at  each  base  by  teams  of  approximately  15 
engineers  or  technicians  contracted  directly  by  the  government  and 
supplied  by  the  radar  contractors  and  the  aircraft  prime  contractors. 
These  teams  would 

•  Conduct  post-flight  debriefings  of  pilots  after  their  normal 
maintenance  debriefings  to  assess  how  well  the  radar  operated. 

•  Observe  and  document — but  not  interfere  with — maintenance 
actions  on  the  flightline  and  in  the  avionics  shop. 

•  Collect  information  on  maintenance  actions  taken  at  the  Ogden 
depot  for  the  F-16  and  at  the  Warner  Robins  depot  for  the  F-15. 

•  Analyze  the  effectiveness  of  maintenance  actions  by  examining 
pilot  inputs  for  flights  that  followed  the  completion  of  mainte¬ 
nance  actions. 

•  Analyze  the  causes  of  ineffective  maintenance  actions  and  pro¬ 
pose  appropriate  remedies. 
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Three  types  of  concerns  delayed  implementation  of  this  plan: 

1.  The  depots  already  had  many  unfunded  opportunities  to 
improve  R&M. 

2.  The  Air  Force  already  had  a  vast  maintenance  data  collection 
system  that  was  underutilized  and  was  being  ignored  by  Stage  1 
efforts. 

3.  The  cost  of  collecting  data  would  exceed  the  cost  of  implement¬ 
ing  some  of  the  already  identified  R&M  improvements. 

Because  of  these  concerns,  the  Air  Staff  asked  the  ASD  Strike  SPO 
to  coordinate  meetings  that  included  the  cognizant  depots  and  SPOs 
and  that  examined  the  need  for  the  proposed  special  data  collection 
efforts  for  the  F-16  and  F-15  radars.  By  June  1982,  these  meetings  had 
established  that  the  Air  Force’s  standard  data  could  not  be  used  to 
assess  radar  R&M  problems  and  determine  their  causes. 

Upon  receiving  this  assessment  from  the  ASD  Strike  SPO,  the  Air 
Staff  began  preparing  program  direction  to  collect  appropriate  R&M 
data  for  the  F-16  and  F-15  radars.  This  direction  was  formalized  in  a 
Program  Management  Direction  (PMD)  sent  to  AFSC  during 
December  1982.  The  AFSC  subsequently  assigned  responsibility  for 
this  PMD  to  the  ASD  Strike  SPO. 

Following  a  series  of  meetings  with  the  cognizant  depots,  SPOs,  and 
ASD  organizations,  the  Strike  SPO  determined  that  the  “appropriate 
data  collection  efforts”  should  have  the  approximate  magnitude  that 
RAND  had  proposed  in  its  Stage  1  specifications.  Accordingly,  the 
ASD  Strike  SPO  in  September  1983  issued  a  Request  for  Proposals 
that  included  most  of  these  specifications. 

The  resulting  F-15/F-16  Radar  R&M  Program  began  its  six-month 
data  collection  efforts  during  June  1984.  The  results  of  the  contrac¬ 
tors’  engineering  analyses  were  presented  to  the  Air  Force  during  May 
1985,  and  the  ASD  briefing  of  the  contractors’  analyses  was  presented 
to  the  Air  Staff  during  September  1985.  At  that  time,  the  Air  Staff 
formed  an  ad  hoc  team  to  prepare  plans  for  Stage  2  attempts  to 
improve  the  R&M  for  the  F-16  and  F-15  radars. 
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